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IMAGE DATA OPTIMISATION USING COMPOSITING ANALYSIS 
Technical Field of the Invention 
The present invention relates generally to the creation of computer-generated 
images and, in particular, to a method and apparatus for creating an image formed by 
5 compositing multiple components. The invention also relates to a computer program 
product including a computer readable medium having recorded thereon a computer 
program for apparatus for creating an image made up by compositing multiple 
components. 

Background Art 

10 Computer generated images are typically made up of many differing components 

or graphical objects which are rendered and composited together to create a final image. 
In recent times, an "opacity channel" (also known as a "matte", an "alpha channel", or 
simply "opacity") has been commonly used. The opacity channel contains information 
regarding the transparent nature of each object. The opacity channel is stored alongside 

15 each instance of a colour, so that, for example, a pixel-based image with opacity stores an 
opacity value as part of the representation of each pixel. An object without explicit 
opacity channel information is typically understood to be fully opaque within some 
defined bounds of the object, and assumed to be completely transparent outside those 
bounds. 

20 A Graphic Object Tree GOB tree offers a systematic means for representing an 

image in terms of its constituent objects and which facilitates later rendering. GOB trees 
typically comprise a plurality of nodes including leaf nodes, unary nodes and binary 
nodes. Nodes of higher degree, or of alternative definition may also be used. A leaf 
node, being the outer most node of a GOB tree, has no descendent nodes and represents a 

25 primitive constituent of an image. Unary nodes represent an operation which modifies 
the pixel data coming out of the part of the tree below the unary operator. Unary nodes 
include such operations as colour conversions, convolutions (blurring etc) and operations 
such as red-eye removal. A binary node typically branches to left and right subtrees, 
wherein each subtree is itself a GOB tree comprising at least one leaf node. Binary nodes 

30 represent an operation which combines the pixel data of its two children to form a single 
result. For example, a binary node may be one of the standard "compositing operators" 
such as OVER, IN, OUT, ATOP and alpha-XOR, examples of which and other are seen 
in Fig. 30. 
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Several of the above types of nodes may be combined to form a GOB tree. An 
example of this is shown in Fig. 31. The result of the left-hand side of the GOB tree may 
be interpreted as a colour converted first image being clipped to spline boundaries defined 
by a path. This construct is composited with a second image. 
5 Although the non-transparent area of a graphical object may of itself be of a 

certain size, it need not be entirely visible in a final image, or only a portion of the object 
may have an effect on the final image. For example, assume an image of a certain size is 
to be rendered on a display. If the image is positioned so that only the top left corner of 
the image is displayed by the display device, the remainder of the image is not displayed. 

10 The final image as displayed on the display device thus comprises the visible portion of 
the image, and the invisible portion in such a case need not be rendered. 

Another way in which only a portion of an object may have an effect is when the 
portion is obscured by another object. For example, a final image to be displayed (or 
rendered) may comprise one or more opaque graphical objects, some of which obscure 

15 other graphical objects. Hence, the obscured objects have no effect on the final image. 

Conventionally, Raster Image Processor (RIP) modules have been utilised to 
process and render graphical images. The performance of an RIP can be affected by a 
number of factors including the volume of pixel data processed, the rendering resolution, 
which is related to the volume of data processed, the complexity of the rendering task, 

20 and the efficiency of the methods employed by the RIP. A number of these factors are 
outside the control of the RIP (e.g. rendering resolution, complexity). Whilst the volume 
of data processed by the RIP is very much affected by such factors as rendering resolution 
and job complexity, there is a minimum volume of data, particularly input pixel data, that 
must be processed in order to produce the required output image. Each pixel in the output 

25 image is derived from a specific source or sources. A pixel may be internally generated, 
or it may be the product of one or more input images used in the render. 

An ideal RIP would only process or generate data which contributes to the final 
rendered image meaning that all computing resources are being efficiently utilised. 
However, no RIP is ideal. The performance of most RIPs is compromised by redundant or 

30 repetitive processing of data. 

Many conventional RIP modules render an object by compositing every pixel 
within the object's bounding box unless the object is known to be fully opaque and 
occupies the whole bounding box. This often involves much unnecessary work since 
objects with irregular boundary outlines must first be padded with transparent pixels out 
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to their bounding boxes, after which the padded pixels have to undergo the full 
compositing operation. 

Disclosure of the Invention 
It is an object of the present invention to substantially overcome, or at least 
ameliorate, one or more disadvantages of existing arrangements. 

According to one aspect of the present invention there is provided a method for 
optimising an expression tree, said expression tree representing a compositing expression 
for compositing an image and comprising a plurality of nodes, each said node of said tree 
representing at least a component of said image or an operation for combining sub- 
expressions of said compositing expression and each said node having a region of the 
image represented by said node, the method comprising the steps of: 

performing a first traversal of said tree to determine opacity information for each 
node of said tree, wherein said opacity information represents combined opacity 
information for a node based on regions associated with the node; and 

optimising said expression tree by performing a second traversal of said tree to 
determine compositing information for at least one node of said tree, wherein said 
compositing information for a node is determined using opacity information associated 
with the node, and wherein said compositing information represents at least obscured 
regions, load regions and regions to be composited for an object associated with the node. 

According to another aspect of the present invention there is provided a method 
for optimising an expression tree, said expression tree representing a compositing 
expression for compositing an image and comprising a plurality of nodes, each said node 
of said tree representing at least a component of said image or an operation for combining 
sub-expressions of said compositing expression and each said node having a region of the 
image represented by said node, the method comprising the steps of: 

performing a first traversal of said tree to determine opacity information for each 
node of said tree, wherein said opacity information represents combined opacity 
information for a node based on regions associated with the node; and 

optimising said expression tree by performing a second traversal of said tree to 
construct a first hierarchical structure for at least one node of said tree, wherein said first 
hierarchical structure is constructed for a node using opacity information associated with 
the node, and wherein said first hierarchical structure represents at least obscured regions, 
load regions and regions to be composited, for an object associated with the node.. 
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According to still another aspect of the present invention there is provided a 
method for optimising an expression tree, said expression tree representing a compositing 
expression for compositing an image and comprising a plurality of nodes, each said node 
5 of said tree representing at least a component of said image or an operation for combining 
sub-expressions of said compositing expression and each said node having a region of the 
image represented by said node, the method comprising the steps of: 

performing a first traversal of said tree to determine opacity information for each 
node of said tree, wherein said opacity information represents combined opacity 
10 information for a node based on regions associated with the node; 

identifying nodes of said tree, for which a first hierarchical structure is required, 

depending on opacity information associated with the node; and 

optimising said expression tree by performing a second traversal of said tree to 

determine compositing information for each node of said tree identified in said first 

15 traversal, wherein said compositing information is determined for a node using opacity 

information associated with the node, and wherein said compositing information 

represents at least obscured regions, load regions and regions to be composited for an 

object associated with the node. 

Brief Description of the Drawings 

20 A number of preferred embodiments of the present invention will now be 

described with reference to the drawings, in which: 

Fig. 1(a) shows a graphical object; 

Fig. 1(b) shows an example of a quadtree describing opacity information of the 
graphical object of Fig. 1(a); 
25 Fig. 2 shows a quadtree configured in accordance with a preferred data structure; 

Fig. 3 shows an optimised quadtree configuration in accordance with the 
preferred data structure; 

Figs. 4 shows an object comprising a fully opaque and fully transparent region; 

Fig. 5 shows the result of a convolution being performed on the object of Fig. 4; 
30 Figs. 6(a) and (b) show an example of the effect on the hidden regions of an 

object following successive convolutions; 

Fig. 7 shows the effect of edge contraction on edge cells; 

Fig. 8(a) shows a fully opaque object covering a whole image render space; 
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Fig. 8(b) shows a quadtree representing the object of Fig. 8(a) after an edge of 
one cell has been applied to the object; 

Figs. 9(a), (b) and (c) show three GOB trees with an obscuring object at different 

depths; 

5 Figs. 10(a) and (b) show a composite worthy object ; 

Fig. 11 shows a composite object where the overlapping of objects is defined by 

rectangular edges; 

Figs. 12(a) and (b) show an example of an opacity quadtree for a small object; 
Fig. 13 is a flowchart showing a method of generating a visibility map for an 
10 image to be rendered; 

Fig. 14 is a flowchart showing a method for fitting rectangles within a 4 x 4 

block using a table look-up; 

Fig. 15 is a flowchart showing a method for determining the scan conversion 
15 priority order of a set of non-overlapping rectangles on a 4 x 4 block; 

Fig. 16 is a flowchart showing an alternative method of optimising a GOB tree; 

Fig. 17 is a flowchart showing a method for determining the obscurance 
quadtrees and reorganising a GOB tree into a right leaning structure in a single traversal 
of the GOB tree; 

20 Fig. 18(a) shows two objects forming a composite object; 

Fig. 18(b) shows the obscurance quadtree corresponding to one of the objects of 

Fig. 18(a); 

Fig. 19 shows an obscurance quadtree overlayed on top of an image object; 

Fig. 20 shows the visibility of an image cell in accordance with an image centric 

25 method; 

Figs. 21(a), (b) and (c) show three examples of the order in which quadrants are 
visited in quadtrees that have undergone various rotations, reflections and shearing; 
Fig. 22 shows an image lying partially outside of an image render space; 
Fig. 23 shows how adjacent quadrants can be merged together to eliminate their 

30 common edges; 

Fig. 24 shows an irregularly shaped visible region that has been converted into a 

series of rectangles; 

Fig. 25 shows the preferred format of a look-up table; 
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Fig. 26 shows an arbitrarily rotated 4x4 block of an obscurance quadtree in 
which the visible cells have been converted into non-overlapping rectangles; 

Fig. 27 shows a block divided into four 2x2 quadrants and the quadrants 
numbered from left to right in accordance with a preferred method; 
5 Fig. 28 shows a set of rectangles processed in accordance with the method of 

Fig. 15; 

Fig. 29 is a schematic block diagram of a general purpose computer upon which 
the preferred embodiment of the present invention can be practiced; and 

Fig. 30 shows examples of objects representing compositing operators; 
10 Fig. 31 shows an example of a GOB tree; 

Figs. 32 shows two objects processed in accordance with a binary operation; 

Fig. 33(a) shows the flow of compositing/obscurance information for a GOB tree 
when a large left operand is composited onto a small right operand where the right 
operand's obscurance quadtree is identical to the parent's obscurance quadtree; 
15 Fig. 33(b) shows the an example of the determination of compositing/obscurance 

information when a large left operand is composited onto a small right operand for an in 
operator where an obscurance quadtree needs to be determined from the nodes opacity 
quadtree and the left operand's bounding box; 

Fig. 34 shows the result of rearranging a right-leaning GOB tree in order to place 
20 some given node at the bottom of the tree; 

Fig. 35 shows a GOB tree with a tagged complex object node; 

Fig. 36(a) shows the GOB tree of Fig. 35 after the children of a tagged node have 
been swapped and the operator being replaced by its opposite; 

Fig. 36(b) shows the GOB tree of Fig. 36(a) after the operator of a left child has 
25 been inverted: 

Fig. 36(c) shows the GOB tree of Fig. 36(b) after having been transformed; 

Fig. 37 shows the GOB tree of Fig. 35 after being processed in accordance with 
the method of Fig. 16; 

Fig. 38 shows the GOB tree of Fig. 36(a) after being rearranged using the 
30 associative property; 

Fig. 39 is a flowchart showing a method of propagating obscurance quadtrees; 

Fig. 40 shows a GOB tree in which one node has been marked as a candidate for 
optimisation; 
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Fig. 41 is a flowchart showing a method for propagating obscurance quadtrees 
using partial opacity information; 

Fig. 42 is a flowchart showing a method for propagating opacity quadtrees for 
each node during the first traversals of the GOB tree in accordance with the method of 
5 Fig. 41; 

Fig. 43 is a flowchart showing a method of rendering an image using quadtrees; 
Fig. 44(a) shows a simple GOB tree along with the GOB tree's associated 
rendering instruction sequence when the left operand of the GOB tree is primitive; 

Fig. 44(b) shows a GOB tree along with the GOB tree's associated rendering 
10 instruction sequence if the left child of the GOB tree is complex; 

Figs. 45(a) and (b) show a GOB tree where a large number of small objects are 

placed over a large object; 

Fig. 46 shows a GOB where a number of small objects share the same subtree as 

a large object; 

15 Fig 47 s hows a right leaning GOB tree comprising of small and large objects; 

Fig. 48 shows a right leaning GOB tree with a large object midway through the 

chain; 

Fig. 49(a) shows a GOB tree processed in accordance with the method of 
propagating opacity quadtrees using bounding boxes; 
20 Fig. 49(b) shows two objects after being combined into the same bounding box; 

Fig. 50 shows a GOB tree where a series of small objects appear in a right biased 

chain of over operations; and 

Fig. 51 is a flowchart showing a method of propagating compositing quadtrees. 
Detailed Description including Best Mode 
25 Where reference is made in any one or more of the accompanying drawings to 

steps and/or features, which have the same reference numerals, those steps and/or features 
have for the purposes of this description the same function(s) or operation(s), unless the 
contrary intention appears. 
1.0 General Overview 
30 A method of rendering an image is described below. The image is formed by 

rendering graphical objects according to a hierarchical GOB tree which represents a 
compositing expression for the image, whereby each of the graphical objects of the GOB 
tree has a predetermined boundary outline. The method uses quadtrees, referred to as 
"obscurance" quadtrees, to perform obscurance analysis on the graphical objects in order 
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to optimise the amount of data processed in rendering the image by excluding the 
obscured obscured regions of objects. A method for propagating the obscurance 
quadtrees for a GOB tree in order to optimise the GOB tree, is also described below. 
Preferably, quadtrees, referred to as "compositing" quadtrees, can also be used to perform 
5 compositing analysis on a GOB tree for optimising compositing operations in order to 
reduce the processing time needed to render an object. A method for propagating the 
compositing quadtrees for a GOB tree, in order to optimise the GOB tree, is also 
described below. Optimising a GOB tree using compositing quadtrees avoids redundant 
processing by limiting a compositing operation to the object's actual boundary outline. 
10 Furthermore, the use of compositing quadtrees can also identify regions that can be 
rendered using simpler and faster load operations, rather than more expensive 
compositing operations. 

1.1 Representing Opacity Information using Quadtrees 

Quadtrees are hierarchical data structures capable of efficiently representing 
15 irregularly shaped regions. Quadtrees work by partitioning an image space into four 
equally sized quadrants, where each quadrant is further and successively partitioned until 
the quadrant represents a homogeneous region of the image, or until some fixed depth is 
reached. Depth in this context refers to the number of levels in the quadtree representing 
the image. 

20 For obscurance analysis, the homogeneous regions of the image are of interest. 

The regions of interest are those corresponding to parts of objects that are of different 
levels of opaqueness (i.e. fully opaque, fully transparent or partially transparent regions). 
These regions are needed in order to take full advantage of possible optimisation 
opportunities that exist in the presence of the various operations (i.e. over, in, out, atop, 

25 etc.). Operators such as over require the fully opaque regions to determine where one 
object hides another, whereas operators such as in and out require the fully transparent 
regions to determine how an object clips out parts of other objects. 

Fig. 1(b) shows an example of a quadtree 100 (hereinafter referred to as "an 
opacity quadtree") describing opacity information of an object 105 shown in Fig. 1(a). 

30 The object 105 is a rectangle divided into two halves by a diagonal. One half (the lower 
left half) is opaque, whilst the other, upper right half is partially opaque. The partitioning 
of the object 105 is also shown in Fig. 1(a) as square blocks under the object 105. As 
seen in Fig. 1(b), the opacity quadtree 100 comprises a number of internal (e.g. 101) and 
leaf (e.g. 103) nodes. Each leaf node (e.g. 103) is assigned a value of 'o\ *-* or 'a' which 
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represent the colours black, white and grey respectively, depending on whether a 
corresponding region in the image space is fully opaque, fully transparent, or partially 
transparent, respectively. If the dimensions of an image space is N x N, and the quadtree 
is of depth q, then a node on level i represents a square region of size N/2 q " x N/2 q '. 

The depth of an opacity quadtree determines the extent to which the image space 
is decomposed. That is, the depth determines the granularity or 'mesh' size of the opacity 
information of each object. The choice of mesh size is a compromise between the 
available computational resources and the effectiveness of the obscurance analysis 
process. The finer the mesh, the more resources are required to process the quadtrees. 
However, since any cell that straddles a fully opaque and a fully transparent region of an 
object is represented as being partially transparent, more opacity information is lost as the 
mesh size gets larger. 
1.2 Configuration off Quadtrees 
1.2.1 Data Structure 

; Opacity quadtrees are generally configured as a collection of linked nodes each 

comprising a value attribute and a set of four pointers. The value attribute indicates 
whether a node is a leaf node, and if so, the opacity value of the image region that the 
node represents. If a node is not a leaf node, then the set of pointers point to the node's 
four immediate children. 
20 Alternatively, each node can store an additional pointer to the node's parent 

node, allowing the quadtree to be traversed in an upward direction as well as down. The 
ability to traverse a quadtree in an upward direction as well as down enables neighbouring 
nodes to be easily located, for example, by following an upward link to the nodes' 
common ancestor and then down towards the desired neighbour. Storing an additional 
25 pointer with each node, which points to a node's parent node, requires approximately four 
traversal of an opacity quadtree in order to visit every node of the tree, independent of 
the depth of the quadtree. However, including an additional pointer to the node's parant 
node does not allow parts of a quadtree to be shared with another quadtree, since a node 
can only point to one parent. Sharing of parts of a quadtree not only saves memory but 
30 also allows quadtrees to be merged more quickly. For example, if a fully transparent 
object is placed over another object, then the quadtree of the composite object simply 
contains a pointer to the quadtree of the second object. 

Preferably, instead of each node of an opacity quadtree containing a set of 4 
pointers to the node's children, only one pointer is stored, which points to the first of the 4 
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children located consecutively in memory. Storing only one pointer in this configuration 
is advantageous when locating a neighbouring node of a particular node, since generally 
the neighbouring node is one of the siblings of the particular node. Thus, by allocating 
the siblings consecutively in memory, neighbouring nodes can generally be located 
5 quickly with minimal search effort. When the neighbouring node is not a sibling, then in 
the worst case it takes q steps to locate the neighbouring node, where q is the depth of the 
quadtree. Therefore, using the preferred quadtree data structure as described above, 
searching for a neighbouring node takes up to q/2 steps. 

As will be described in more detail later in this document, preferably two 

10 additional attributes can be assigned to each node. The two additional attributes are an 
"area" and a "number of nodes" attribute. The area attribute stores the total area of the 
fully opaque region and the "number of nodes" attribute stores the total number of nodes 
found in a particular subtree at or below the node. These additional attributes can be used 
to determine whether the object represented by a particular quadtree is a worthy candidate 

15 for obscuring other objects. 

Fig. 2 shows a quadtree configured using the preferred data structure, where 
"NW" refers to a North West region of the image space, "NE" refers to a North East 
region of the image space, "SW" refers to a South West region of the image space, and 
"SE" refers to a South East region of the image space. 

20 1 .2.2 Optimising the data structure 

In simulations carried out by the inventor in order to study characteristics of the 
quadtree representations of randomly generated axis-parallel rectangular objects, it was 
found that approximately half of the nodes in the quadtrees are leaf nodes residing at the 
bottom most level (i.e. level 0). 

25 Therefore, if the processing of quadtrees can be optimised, even if it is only done 

on the bottom most level of the trees, the result is a greatly reduced processing time. 
Although the above-mentioned simulations only involved rectangular objects, the 
inventor hypothesised that for other objects that are not composed of axis-parallel edges, 
the proportion of nodes on the bottom level would be higher, and hence the resulting 

30 reduction in processing time would be even greater. 

Preferably, nodes residing on the bottom most level of quadtrees are stored 
compactly. For nodes on the second bottom most level (ie. level 1) that have children, 
instead of allocating memory for the four child nodes and storing a pointer to the first of 
these child nodes, the values of the four child nodes can be packed together into a single 
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8-bit number. Since the preferred quadtree data structure contains nodes of three distinct 
values, two bits are needed to represent the value of a single node, and four values can be 
packed into 8 bits. 

Fig. 3 shows an optimised quadtree configuration for nodes on the second 
5 bottom most level (ie. level 1) of the quadtree 300 that have children. As can be seen in 
Fig. 3, nodes residing on level 1 (e.g. 301) that do not have children (ie. leaf nodes) are 
also stored using the above described packed representation. In these cases, the values of 
the four quadrants in the packed 8-bit number are identical. Therefore, nodes on level 1 
are preferably decomposed into their 4 quadrants regardless of whether the quadrants 

10 hav.e identical values or not. 

As with the node data structure shown in Fig. 2, the new packed nodes can also 
store the attributes "area" and "number of nodes". These attributes can be defined in the 
same way as in the node shown in Fig. 2. However, since the packed nodes are located at 
the bottom of the tree, where each quadrant is of unit area, the area reduces to the number 
15 of fully opaque quadrants, and the "number of nodes" is one if the four quadrants have 
identical values, and five otherwise. 

The packed data representation not only requires smaller storage space but also 
leads to faster processing since the processing of nodes on the bottom level can be 
performed simultaneously on four siblings at a time. Furthermore, the quadtree node data 
20 processing can be performed using a simple table look-up requiring very little processing 
time. Since each packed data block is 8 bits wide, an operation involving two quadtrees 
(eg..merge) requires a look-up table of size 256 x 256 = 65536 bytes which is a relatively 
insignificant memory demand. 

: - Although the optimised quadtree configuration shown in Fig. 3 can be extended 
25 to nodes residing on levels other than 0, the benefit of doing so is minimal since the 
packed data structure can only be used on nodes whose child quadrants are all leaf nodes, 
which rarely occurs on a level other than the bottom most level. Often one or more 
quadrants are decomposed into smaller cells. 
2.0 Propagating Obscurance Quadtrees 

30 2.1 Overview 

Obscurance analysis requires knowledge of the opacity information of each 
object, in the form of opacity quadtrees, so that regions where objects have been hidden 
or clipped out by other regions can be identified. These obscured regions are generally 
irregular in shape and can also be represented using quadtrees, referred to as obscurance 
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quadtrees. Unlike opacity quadtrees, obscurance quadtrees preferably contain only two 
distinct node values instead of three. The two distict node values being *1* where the 
object is hidden, and *0* where the object is visible. Fig. 18(a) shows two objects 1801 
and 1803. Object 1803 is partially transparent in one region 1805 and fully opaque in 
5 another region 1807. Further, object 1803 is over object 1801. The obscurance quadtree 
1809 corresponding to object 1801 is shown in Fig. 18(b). As shown by the obscurance 
quadtree 1809, object 1801 is visible in all parts of level 2 quadrants 1 and 3, indicated by 
nodes 1811 and 1813, respectively. Further, object 1801 is hidden in level 1 quadrant as 
indicated by node 1815. 

10 Fig. 39 is a flowchart showing a method of propagating obscurance quadtrees in 

order to optimise a GOB tree representing an image. The method comprises the 
computation of opacity quadtrees and the computation of obscurance quadtrees. The 
process begins at step 3901, where the computation of opacity quadtrees is preferably 
performed as a bottom-up GOB tree traversal process where the opacity quadtree of each 

15 graphics primitive is determined and propagated up the GOB tree. At each compositing 
node, an opacity quadtree is constructed based on the opacity quadtrees of the node's 
child operands and the type of compositing operator involved. At the next step 3403, 
obscurance quadtrees are computed from the opacity quadtrees. As each node in the 
GOB tree inherits the obscured regions of the node's parent node, since anything that 

20 obscures a composited object also obscures the individual objects that make up the 
composite, the obscurance quadtrees are propagated in a downwards GOB tree traversal. 
The process concludes when the final obscurance quadtree arrives at each leaf node. The 
final obscurance quadtree at each leaf node can then be used to limit to the amount of 
processing required to render the graphics object corresponding to the particular leaf 

25 node. 

As large parts of a typical GOB tree comprise of binary compositing operators, a 
method for propagating opacity and obscurance quadtrees can be described through the 
various binary operators discussed above. 
2,2 Propagating Opacity Quadtrees 
30 The opacity quadtree of a composite object can be readily constructed through a 

merge operation between the opacity quadtrees of the individual objects. The merge 
operation involves traversing both input quadtrees and constructing the resultant quadtree 
in parallel. The value of a node in the resultant quadtree is determined by the 
corresponding node values of the input quadtrees and the type of compositing operator 
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involved. Table 1, below, shows resultant node values as a result of input node values. 
The columns 'L\ 'R' and 'Resultant' denote the node values of the left and right input, 
and the resultant quadtrees, respectively. 



Table 1 
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As seen in Table 1, the entries for each operator have been divided into two 
parts: those corresponding to regions where one or both of the operands are fully 
transparent, and those for the remaining regions. In the latter, the different combinations 
of node values in the input quadtrees are listed with the corresponding node value of the 
resultant quadtree, whereas in the former, they can be represented more compactly. In 
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Table 1, L denotes the value (o/-/a) of the left operand's region, R denotes the value (o/- 
/ct) of the right operand's region, and X denotes a 'don't care' value. 
23 Propagating Obscurance Quadtrees 

As discussed above, obscurance quadtrees preferably have 2 distinct node 
values: namely '1* denoting a hidden region, and *0* denoting a visible region. Using 
standard set notation, if Q P represents the union of all the ' 1 ' (hidden) regions of the 
obscurance quadtree of a composite object, then the '1' (hidden) regions of its left and 
right operands can be stated as shown in Table 2 below, where Q L and Q R represent the 
opacity quadtrees of the left and right operands, respectively: 



Table 2 



Operator 


Left operand 


Right operand 


Over 


Qp 


Qpu{Q L ='o'} 


Out 


Qp^{Qr='o'} 


Qp^{Q R = 'o'orQ L ='-'} 


In 


Qp^{Qr = '-'} 


Qp^{QR='o'orQ L ='-'} 


Atop 


Qp^{Qr='-'} 


Qp '-'{Ql = 'o' and Q R = 'o'} 


Xor 


Qpu{Q L ='o'andQ R = 


Qp ^{Ql = 'o' and Q R = 'o'} 




'o'} 




Plusc, plusw, 


Qp 


Qp 


rops 







Note that each obscured region as defined in Table 1 for the left operand can be 
15 unioned with the set {Q L = '-*} without affecting the final rendered image. This can be 
performed since adding the set {Q L = *-*} represents the 'empty' region of the object, and 
hence adding this set to the hidden region simply forces a rendering system not to render 
the operand where the operand is empty. Likewise, the set {Q R = '-'} can be added to the 
obscured region of the right operand without affecting a rendered image. However, such 
20 an addition can affect the amount of work needed to compute the obscurance quadtrees. 
In most cases, the addition of {Ql = *-'} and {Qr = '-*} to the obscured region can lead to 
unnecessary computations. For example, the left operand for the over operator, has an 
obscurance quadtree of Qp *u {Q L = *-*} requiring one merge operation, instead of the 
original Q P which does not require any computation. 
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In other cases, the addition of {Q L = '-*} and {Qr = '-'} to the obscured region 
results in the amount of work involved being slightly less. For example, the obscurance 
quadtree for a right operand of the out operator (i.e Q P u{Q R = 'o' or Q L = '-' W {Qr = 
'-'}) can be expressed as Q P u{Q R = 'o'} u {Qr= '-'} u {Ql= '-'}• 

The expression Q P u{Q R = 'o'} u {Qr = '-'} u {Ql = '-'} can be determined 
more efficiently than the original, Q P u{Q R = V or Ql = '-'}, since there are more cases 
where the new expression can be determined from the knowledge of only the value of Q R , 
namely where {Q R = '-'}. As will be discussed in Section 4.1, a merge operation can be 
performed more efficiently on parts of quadtrees where the results are dependent on only 
the values of one quadtree rather than both. However, in most cases the addition of {Ql = 
'-'} and {Qr = '-'} to the obscured region is of negligible use since it has a detrimental 
impact on the processing of partial opacity information which will be discussed in Section 
2.4. The computation of Q P u{Qr = 'o' or Ql = «-'} only requires information about the 
fully opaque region of the right operand (in addition to the fully transparent region of the 
left operand), whereas the expression Q P u{Q R = V or Ql = *-'} <~» {Qr = '-*} requires 
information about the fully transparent region of the right operand as well. The one area 
where the addition of {Ql = *-'} and {Qr = '-'} to the obscured region can result in 
optimised processing is in special cases where the right operand is known to be fully 
opaque, which will be examined in the section 2.3.1, below. 
2.3.1 Processing Fully Opaque Operands 

For the in and out operators, some analysis can be possible during the 
determination of an obscurance quadtree when the right operand is fully opaque. In such 
cases, the right operand does not affect the composite image at all, but is merely used to 
clip in or out parts of the left operand where they intersect. Thus, the obscurance 
quadtree for the right operand is effectively a tree with a single hidden node that spans the 
entire render space in which the image is defined. This result can be formally deduced 
from Table 2, by first adding the set {Qr = '-'} to the obscurance quadtree as discussed 
previously, resulting in Q P u{Q R = 'o'} u {Ql = '-'} ^ {Qr = '-'}• 

If the right operand is fully opaque, then the set {Qr = 'o'} kj {Qr = '-'} covers 
the entire render space. Therefore, Q P u{Q R = 'o' or Ql = '-'} ^ {Qr = *-*> s P ans ±e 
entire render space as well. 

2.4 Processing Partial Opacity Information 
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The above description of propagating opacity and obscurance, has assumed that 
all such information is required and is available. However, there are situations where 
determination of the necessary obscurance information does not require the determination 
of the full opacity information of all objects, and hence further optimisation can be made 
to the amount of processing involved in rendering an image. For example, at the root 
node of a GOB tree being processed, an opacity quadtree, is not required. If, in addition 
the node is an over operator, then the opacity quadtree for the right operand is also not 
required, and only the fully opaque regions of the left operand are required. To take 
advantage of the possible reduction in processing where propagating opacity and 
obscurance information is not necessary, the minimum opacity information required when 
encountering different types of compositing operations and different requests for opacity 
information needs to be identified. 

There are two main types of partial opacity information that are of interest in 
obscurance analysis, which are referred to as inner_op and outer_max. As discussed 
above, the preferred opacity quadtree configuration contains nodes with 3 distinct values, 
namely 

*-* representing fully transparent objects; 
'a 5 representing partially transparent objects; and 
4 o' representing fully opaque objects. 
inner _op represents the case where it is necessary to accurately identify regions 
that are fully opaque, but it is not necessary to identify regions that are fully or partially 
transparent. It is of no concern whether a fully transparent region is incorrectly labelled 
as partially transparent, or vice versa. Similarly, outer jnax denotes the case where it is 
necessary to identify regions that are fully transparent, but it is not necessary to identify 
regions that are fully opaque or partially transparent. 
2.4. 1 Partial information requested by a parent node. 

Table 3, below, shows what opacity information is required from the left and 
right child operands of a binary operator when only partial opacity information needs to 
be determined for the parent node, for each compositing operator. This information is 
used to compute the opacity quadtrees of nodes during a first traversal of the GOB tree. 
The corresponding entries in the L and R columns of the table indicate what opacity 
information is required from the left and right operands. 

Table 3 
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Operator 


jr arem 


L 


R 


over, in, 


It VI 1^1 


Innerjop 


innerjop 


plusc, rops 


outer jnax 


Outer jnax 


outer jnax 


out 




Inner_op 


outer jnax 




outer _max 


Outer max 


innerjop 


atop 


1YIYIPY on 

ouier ffiiMx 




innerjop 
outer jnax 


xor 


innerjop 


both 


Both 




outer jnax 


both 


Both 


plusw 


innerjop 


both 


Both 




outer jnax 


Outer jnax 


outer jnax 
— 



) 



Table 3 shows that with the exception of the xor and to some extent the plusw 
operators, only partial opacity information is required from the child operands when only 
partial information is requested at the parent node. 

242 Information required for performing obscurance analysis on operands. 

Section 2.4.1 examined what opacity information is required from child operands 
to partially determine the opacity information of the parent compositing nodes. Tins 
information can be used to compute the opacity quadtrees of nodes during the first 
traversal of the GOB tree. However, opacity quadtrees determined as indicated in Table 3 
do not contain sufficient information for use in constructing obscurance quadtrees m the 
second traversal of the GOB tree since the process of building obscurance quadtrees 
makes use of clipping information stored in the opacity quadtrees that may not have been 
included when the opacity quadtrees were constructed with partial informal m the first 
pass. 

Table 3 can be combined with Table 4, below, which lists the opacity 
information from each operand required to capture the clipping information. Each 
operator in Table 4 contains two rows of entries. The letter L or R in the column 
"Perform opt. on" indicates whether it is necessary to perform obscurance analysis (ie. 
whether parts of objects that do not need to be rendered should be clipped out) on the left 
or right operand. Typically, obscurance analysis is only applied to 'complex' objects 
such as images and perhaps edge blends, rather than on simple graphics primitives since 
the time spent on analysis may not be compensated by the benefits of the optimisation. 
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The entries in the columns "L" and "R" of Table 4 indicate what opacity 
information is required to do the analysis. A denotes no information is required. For 
example, for the over operator, no opacity information is required from either operand to 
optimise the left operand since the left operand is on top and hence is not obscured by the 
right operand, whereas to optimise the right operand, information about where the left 
operand is fully opaque is required. 

Table 4 



10 



15 



20 



Operator 


Perform opt. on 


L 


R 


over 


L 








R 


innerop 




out 


L 




innerjop 




R 


outer max 


inner op 


in 


L 




outer max 




R 


outer max 


inner op 


atop 


L 




outer _max 




R 


inner op 


inner op 


xor 


L 


inner_op 


inner op 




R 


inner op 


inner op 


plusc, rops, 


L 






plusw 


R 







The contents of Table 4 can be obtained from Table 2, as follows: 

(i) For optimising the Left operand, the column named "Left operand" of 
Table 2 is used. 

(ii) Check for the occurrence of Q L = 'o\ Q L = 4 -\ Qr = *o\ and Q R = *-\ 
The opacity information required depends on which of the above terms 
are present where: 



implies that the inner _op is required from the left operand; 
implies that the outer _max is required from the left operand; 
implies that the inner _op is required from the right operand; 

implies that the outer jnax is required from the right 



25 



Ql='o' 
Ql='-' 
Qr='o' 
and 

Qr='-' 
operand. 

Similarly, the opacity information required when optimising the right operand 
can be determined by searching for the above terms from the "Right operand" column of 
Table 2. 

2.5 Propagating obscurance quadtrees using partial opacity information 
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Fig 41 is a flowchart showing a method for propagating obscurance quadtrees 
using partial opacity information, in order to optimise a GOB tree representing an image. 
In order to implement selective analysis (i.e. one in which only complex objects are 
optimised) it is necessary to increase the number of traversal of the GOB tree over the 
method for propagating obscurance quadtrees discussed above, from two to three. This is 
because an additional step is needed to locate where the complex objects reside in the 
GOB tree. The process begins at step 4101, where a bottom-up traversal of the GOB tree 
is carried out to determine where complex objects (eg. images) are located. Each leaf 
node corresponding to such an object is marked as a candidate for optimisation. This 
marking is propagated upwards, such that every parent node whose one or more children 
are candidates is itself also a candidate for optimisation. At the next step 4103, a second 
traversal of the GOB tree is carried out whereby the traversal comprises two sub-steps (ie. 
a top-down process followed by a bottom-up process). The top-down process involves 
examining each parent node for analysis markings generated in the first traversal, at step 
4101 to determine what opacity information is required from the child nodes. No opacity 
information is needed at the root node and if one or more child nodes of the root node are 
marked as candidates for optimisation, the process looks up Table 4 and records what 
opacity information is required for each child node. The opacity information required for 
each child node is propagated down to each child node. The bottom-up process at step 
4103 is a recursive process in which for each parent node in the GOB tree, the opacity 
information required is used to look up Table 3 to determine what information is needed 
at each of the child nodes. In addition, if one or more child nodes has been marked as a 
candidate for optimisation, Table 4 is also looked up to determine what additional 
information is required. The combined requirements are then propagated down the GOB 
tree The process continues at step 4105, where having identified what opacity 
information is required at each leaf node, an opacity quadtree is constructed for each leaf 
node and propagated upwards in a post order traversal to determine the opacity quadtree 
of the parent node. The method of propagating the opacity trees will be explained in 
more detail later in this document. The process concludes at step 4107, where a top-down 
traversal of the GOB tree is carried out. At each node in the top-down traversal, an 
obscurance quadtree is first generated from the node's opacity quadtree and the opacity 
quadtrees of the node's sibling. The obscurance quadtree is then combined with that of 
the node's parent obscurance quadtree to propogate a final obscurance quadtree for the 
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node. The process concludes when the obscurance quadtree for the last leaf node is 
determined. 

3.0 Convolutions 

A convolution operator has the effect of reducing the fully opaque region of an 
5 object by an amount equal to the convolution radius, and if the object has holes, reducing 
fully transparent regions of the object within the holes. Fig. 4 shows an object 400 having 
a fully opaque 401 and fully transparent 403 region. Fig. 5 shows the effects of 
convolution on the fully opaque 401 and fully transparent 403 regions of the object 400, 
where solid and grey areas denote the fully opaque and partially transparent regions of 

10 the convolved object 500, respectively. The convolved object 500, as seen in Fig. 5, 
shows the object 400 after convolution has been performed on the object 400. As seen in 
Fig 5, the fully transparent 403 and fully opaque 401 regions of the object 400 are 
contracted by an amount equal to the convolution radius, with the areas 501 and 503 
around the edges of the fully opaque 401 and fully transparent 403 regions, respectively, 

15 being transformed to partially transparent regions. 

3.1 Propagating Opacity Quadtrees through a Convolution Operator 

As discussed above, a convolution operator has the effect of reducing the fully 
transparent and fully opaque regions of an object. Thus, in the second traversal of the 
20 GOB tree, as at step 4103 to 4105 of Fig. 41, when opacity quadtrees are generated, the 
fully opaque and fully transparent regions of an object need to be contracted as the 
opacity quadtree of the object is propagated up through a convolution operator. The 
amount of contraction is equal to the radius of the convolution. The contraction is 
preferably rounded up to the nearest integral number of mesh cells, since generally a 
* 25 discrete mesh size is being processed. 

When there are more than one convolution operators being processed, instead of 
contracting the regions in a number of separate discrete steps, the regions can be 
combined all together, which can lead to a smaller total contraction size. However, this 
can only be done when the convolution operators are applied in succession, without any 
30 other intervening operators. 

3.2 Propagating Obscurance Quadtrees through a Convolution Operator 

In the third traversal of a GOB tree, as at step 4107 of Fig. 41, obscurance 
quadtrees are generated and propagated downwards. The obscurance quadtree of a node 
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indicates regions where the corresponding object is hidden or clipped by other objects. 
When propagating an obscurance quadtree through a convolution operator, any hidden 
regions need to be contracted by an amount equal to the radius of the convolution since an 
area of an object that is hidden can contribute to a pixel in a nearby visible region, due to 

5 the convolution operation. 

When a succession of convolution operators are applied to an object their effects 
are cumulative. Fig. 6 shows an example of the effect on the hidden regions of an object 
A following successive convolutions. A fully opaque object 'A' is placed over another 
object 'B\ thus obscuring part of object B. The GOB tree 600 representing the composite 

10 object is shown in Fig. 6. In this example, object B has undergone two successive 
convolution operations before being obscured by object A. Consider three equally spaced 
points X, Y, and Z of B, as shown in the Fig. 6, where y and z lie in an obscured region 
601. Let the radius of both convolution operators be equal to the spacing between the x 
and y. Let x* and y 9 be the image values of B at points x and y after the application of the 

15 first convolution operator. Likewise let jc" be the image value of B at point x after the 
application of both convolution operators. 



then x 9 =f\(x 9 y) 

y =M*>y> z) 

20 and x"=f 2 (x\y 9 ) 

=f2<f\(x,y),f\(x,y> z)) 



where /i and/ 2 represent the convolution operations. Thus x" is a function of the original 
image values of B at all three points x, y and z, and hence the obscurance quadtree of B 
25 must not include point z, which is located within a distance equal to the sum of the 
convolution radii. 

As in the propagation of opacity quadtrees, the effects of a succession of 
convolution operators can be lumped together into one single contraction of the 
obscurance quadtree. Again however, this can only be done when there are no other 
30 intervening operators present. 
4.0 Processing Quadtrees 

Two methods are described below for processing the opacity and obscurance 
information stored in the form of quadtrees. The first method is for merging two 
quadtrees together and is used to propogate the opacity quadtree of a parent node from 
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those of the node's children due to the application of a binary operator. The first method 
can also be used for determining each operand's obscurance quadtree from the operand's 
opacity quadtree and the opacity quadtrees of its siblings. The second method is used for 
contracting certain regions in a given quadtree due to convolutions. 
5 4.1 Merging Opacity Quadtrees 

Fig. 42 is a flowchart showing the preferred method for propagating opacity 
quadtrees for each node, during the first traversals of the GOB tree at step 4101 of Fig. 
41. The method works by traversing two input opacity quadtrees for two nodes of the 
GOB tree and constructing the resulting opacity quadtrees in parallel. The process begins 

10 at step 4200, where a pair of corresponding nodes in the two input opacity quadtrees is 
selected. At the next step 4201, if the current nodes in both input opacity quadtrees are 
leaf nodes, then the value stored at the nodes are used to look up Table 5, at step 4203, in 
order to determine the corresponding node value of the resultant opacity quadtree, based 
on the type of binary operator associated with the GOB tree nodes. Otherwise, the 

15 process proceeds to step 4221, where if the current nodes in both input opacity quadtrees 
have children, then each of these children are selected, at step 4205, in turn and the 
process returns to step 4201. If one of the nodes is a leaf node at step 4221, then the 
process proceeds to step 4223, where the node value at the leaf node is used to look up a 
shaded entry in Table 5 in order to determine the corresponding node value of the 

20 resultant opacity quadtree. 

The methods used for the different types of compositing operators differ in the 

processing of cases where the current node of one input opacity quadtree is a leaf node 

whilst the current node of the other input opacity quadtree is not. Special processing of 

these cases is required since optimisation can be possible as can be seen from a 

25 restatement of Table 1, in Table 5 as follows: 

Table 5 



Operation 


L 


R 


Resultant 


Over, 




R 


R 


plusc, 


o 


X 


o 


rops 


L 




L 




X 


o 


o 




a 


a 


a 


Out 




X 






X 


o 






L 




L 




a 


a 


a 




O 


a 


a 


In 




X 






O 


R 


R 




X 








L 


o 


L 




a 


a 


a 
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Atop 


X 


R 


R 


Xor 


- 


R 


R 




a 


X 


a 




L 


- 


L 




X 


a 


a 




O 


o 


- 


Plusw 




R 


R 




a 


X 


a 




L 




L 




X 


a 


a 




O 


o 


a 



Table 5 is a more compact representation of the same data in Table 1, where 
entries with the same input values for the left or right operands have been grouped 
together wherever possible. Table 5 is useful since for the cases shown shaded grey, the 
5 resultant values can be determined from the value of only one of the operands. For 
example, the second row (ie. for over) states that if the left operand of a branch is fully 
opaque, then the resultant value is fully opaque regardless of the value of the right 
operand. Similarly, the first row states that if the left operand is fully transparent, then the 
resultant value is the same as that of the right operand, whatever the value of the right 

10 operand is. Table 5 allows the merge operation input opacity quadtrees to performed 
more efficiently when handling cases where the current node of one input opacity 
quadtree is a leaf node whilst the current node of the other input opacity quadtree is not. 

After step 4223, the process proceeds to the next step 4207, where if the value of 
the leaf node matches one of the shaded cases shown in Table 5, then the process 

15 continues to step 4209, where there are two possible outcomes when merging input 
opacity quadtrees. At step 4209, if the resultant value as indicated in Table 5 is a fixed 
value (ie. o/-/a) then the output resultant quadtree is appended with a leaf node at the next 
step 4211, or else the sub-tree below the non-leaf current input node is appended to the 
output quadtree, at step 4213. In either case, it is not necessary to process any part of the 

20 input opacity quadtrees below the current nodes. After step 4213, the process concludes 
at step 4217, when all nodes of the input opacity quadtrees have been visited. Otherwise 
the process proceeds to step 4200. 

In contrast, if the value of the leaf node of one opacity quadtree is not one of the 
shaded cases shown in Table 5, at step 4207, then the region represented by this leaf node 

25 is split into four smaller quadrants, at step 4215, and the process returns to step 4205. 
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After either step 4203 or step 4211, the process proceeds to step 4225, where if 
each set of sibling nodes of the current node have not been processed, the process returns 
to step 4200. Otherwise, the process proceeds to step 4218, where a check is performed 
to determine if the resulting nodes in the output quadtree are leaf nodes and that they have 
5 been assigned identical node values. If so, then the process proceeds to step 4219, where 
the four sibling output nodes are combined into a single larger leaf nodes with the same 
node value. Otherwise, the process proceeds to step 4217, where the process concludes if 
all nodes of the input opacity quadtrees have been visited. If all nodes of the input opacity 
quadtrees have not been visited, the process proceeds to step 4200. 

10 From the above description, it can be seen that the complexity of the method for 

propagating the opacity quadtrees, in terms of the number of nodes that need to be visited, 
is directly related to the number of nodes in the input opacity quadtrees. An upper bound 
on the number of nodes that need to be visited is the total number of nodes in both input 
quadtrees. Fewer nodes will be visited if there exists corresponding pairs of nodes in the 

15 input opacity quadtrees such that when one of the nodes is a leaf node and the other is not, 
the value of the leaf node is among the cases shown in grey in Table 5. 
4.2 Merging Obscurance Quadtrees 

The propagation of an obscurance quadtree through a binary compositing 
operator in the second traversal of the GOB tree, at step 4103 of the method of Fig. 41, 

20 can involve up to three merge operations. For the over operator however, only one merge 
operation is needed. This can be seen from Table 2. For the over operator, no processing 
is required to construct the obscurance quadtree for the left operand since the obscurance 
quadtree will be identical to that of the parent node, and only one merge operation is 
needed to compute the expression Qp u{Ql = *o*}. 

25 In contrast, for the out operator, three merge operations are required in the 

propagation of an obscurance quadtree in the second traversal of the GOB tree, at step 
4103 of the method of Fig. 41. One operation for the left operand, and two operations to 
compute the expression Q P ^{Qr = 'o' or Q L = '-'} for the right, which comprises of first 
a merge between the opacity quadtrees Q L and Q R , and then a merge with the parent 

30 obscurance quadtree Qp. 

Since the expressions for computing the obscurance quadtrees shown in Table 2 
comprise mixtures of opacity quadtrees (Q L and Q R ) and the obscurance quadtree (Q P ), a 
number of merge methods are required for dealing with the different combinations of 
inputs and outputs. The merge methods comprise the merging of two opacity quadtrees to 
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obtain an obscurance quadtree, the merging of an opacity and an obscurance quadtree to 
obtain an obscurance quadtree, and the merging of two obscurance quadtrees to obtam an 

obscurance quadtree. 

The different methods are all variants of the method for propagating two opacny 
5 quadtrees in order to produce another opacity quadtree as discussed in Section 4.1, wtth 
one significant difference. Since the outputs of the merge methods are obscurance 
quadtrees, some conversion can be necessary if one or both of the inputs are opactty 
quadtrees. This conversion is undesirable since it means that aU input opacity quadtrees 
must be fully explored. For example, in the computation of the union between an 
,0 obscurance quadtree Qr and the set {Ql - V). if Qp is an empty se, (i.e. a quadtree wtth 
a single "visible- root node), then what is otherwise a trivial operation requrres a full 
traversal of Ql to convert all V nodes to -hidden' and all •-• and •«■ nodes to "vistble-. 

In order to overcome the conversion if one or both of the inputs of a merge 
method are opacity quadtrees, obscurance quadtrees can be allowed to carry the same 
15 three possible node values (i.e. V and V) as opacity quadtrees. V nodes for 
obscurance quadtrees are then treated as "hidden- and the other nodes as 'vtstble-. 
However, the disadvantage of allowing obscurance quadtrees to carry the same three node 
values as opacity quadtrees is that these obscurance quadtrees can be slightly more 
complex (ie. contain more nodes) man their two-value equivalents, since mergmg stblmg 
20 quadrants containing different (■-' and •«•) but otherwise equivalent values into a smgle 

node is unable to be done. 

A conversion referred to as a - toy- conversion can be performed if one or both 
of the inputs of a merge method are opacity quadtrees. That is, the conversion of opactty 
quadtree nodes into the 2-value format is delayed until the nodes are visited dunng 
25 subsequent merge operations. Nodes that do no. need to be visited are left as they are. 
For example, computing the set Qp u <Ql - V> when Qp is an empty se, simply returns 
the original quadtree Ql. Only when Ql is merged with another quadtree mat reqtures 
some of the other quadtree's nodes to be explored are those nodes converted mto the two- 
value format. Therefore, the number of nodes in the quadtree can be reduced wtthout 
30 spending any additional time exploring nodes unnecessarily. 

4 3 Convolution 

To perform the contraction of fully opaque and fully transparent regions as a 
result of propagating opacity quadtrees through a convolution operator, a method for 
identifying "edge- cells in the quadtrees is required. Edge cells are cells tha, border 
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another cell of a different value. Thus, an essential ingredient of any edge finding method 
is to explore neighbouring nodes. There are two main methods for locating neighbouring 
nodes, namely a bottom-up or a top-down. 

The bottom-up method works by traversing along upward links of a GOB tree 
5 towards a common ancestor and then downwards to locate the desired neighbour. The 
bottom-up method requires each node to store a link to the parent. Since it takes an 
average of 4 steps through the quadtree to locate a neighbouring node, independent of the 
depth of the tree, the bottom-up traversal method has an average complexity that is linear 
to the number of nodes in the quadtree being processed. However, the worst case 

10 complexity is quadratic. 

The top-down edge finding method on the other hand explores nodes in a top- 
down manner, and hence does not require the storage of parent links. The top-down 
method works by maintaining a set of pointers to the 8 neighbouring nodes of each node 
as the opacity quadtree is traversed. The top-down method is more efficient than its 

15 bottom-up counterpart due to the maintenance of the set of neighbour pointers, which 
means neighbouring nodes can be located with less effort. The top-down method has a 
worst case linear complexity as opposed to the average linear complexity of the bottom- 
up method. 

4.3. 1 Top-down edge finding method 
20 As discussed above, a top-down edge finding method works by maintaining a set 

of pointers to the neighbouring nodes of each node as the opacity quadtree is traversed in 
a top-down manner. 

Two nodes that are not descendants of one another are said to be neighbouring 
nodes if their corresponding image regions share a common boundary, even if only at a 

25 single point. Note that this definition applies to all nodes in an opacity quadtree whether 
they are leaf or non-leaf nodes, and that a pair of neighbouring nodes can be located on 
different levels (ie. the image regions represented by the nodes are of different sizes). 
Neighbouring cells are defined as being neighbouring leaf nodes. 

In order to find edges of an object, it is necessary to locate neighbouring cells. 

30 In a bottom-up method, locating neighbouring cells is done by traversing the opacity 
quadtree and at each leaf node, performing a search to locate neighbouring cells. In the 
top-down method, a set of pointers is maintained to the neighbouring nodes so that less 
search is needed. The set of pointers indexes the set of neighbouring nodes in the 8 
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opacity quadtrees is bounded above by a linear function of the object's perimeter. Since 
the perimeter of an object remains roughly the same after an edge contraction (i.e. a 
change of a factor of 2 or less), the processing required is thus linear in the perimeter of 
the original object. 

5 Edge contraction is considerably more computationally expensive when 

compared to the opacity quadtree propagation method. The inventors have shown that 
edge contraction is of an order of magnitude slower than quadtree merging. Another 
contributing factor to the speed of the edge contraction method is the fact that a new and 
complete quadtree needs to be constructed, whereas in merge operations, parts of the 

10 input quadtrees can often be reused. 

4.3.4. Region contraction of more than one cell width 

The edge finding and contraction methods discussed in the previous sections are 
only applicable when the convolution radius is less than or equal to 1 cell width. 
Convolutions with larger radii can be generally handled by applying the region 

15 contraction methods a number of times in succession which can be time consuming if the 
radii are very large, but such cases are rare. 

Alternatively, radii of arbitrary sizes can be processed. However, in this 
instance, the top-down edge finding method discussed above requires the set pointers to 
index all nodes lying within a distance equal to the convolution radius from each node, 

20 and processing time grows rapidly as the radius gets larger. 

For the special case where the convolution radius is 2 cell widths, the packed 
data structure introduced in Section 1 .3.2 can be utilised to perform the region contraction 
with the same efficiency as the method for handling 1 cell width region contraction. In 
this instance, only 8 pointers need be maintained to the eight neighbouring packed cells of 

25 each node. 

5.0 Selective Obscurance Analysis 

During the propagation of quadtrees, there is a constant need for merging 
quadtrees when compositing operations are encountered. As discussed above, the 
merging of quadtrees occurs during both the second traversal of the GOB tree, as at step 

30 4103, when opacity quadtrees are constructed, and the third traversal, as at step 4107, 
when the obscurance quadtrees are constructed. 

The number of merge operations is a direct function of the number of objects in a 
GOB tree. Therefore, for a large GOB tree, the time spent on obscurance analysis can be 
quite significant. If there is a high degree of obscurance between objects, then the time 
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ratio of the object's area to the number of nodes in the opacity quadtree representing the 
object. 

5.1.1 Depth 

Depth is another factor that affects the worthiness of a potential obscurer. A 
5 moderately sized object located deep down a GOB tree representing an image is not as 
favourable an obscurer as a similar object located further up the tree, since the object 
requires much processing to propagate the object's opacity information up where the 
opacity will be finally used. Preferably, the 'depth' of an obscurer should not be an 
object's absolute depth, but rather the object's depth relative to the objects being 
10 obscured. For example, three GOB trees GOB 901, 903, and 905 are shown in Figs. 9(a), 
(b) and (c), respectively. 

In the examples of Figs. 9(a), (b) and (c), object A is large and hence is a 
potential obscurer. Object B is an image being obscured. In Fig. 9(a) with tree 901, both 
objects A and B are located in the shallow part of the GOB tree 901, and hence little 
15 processing is needed to obscure object A on B. In Fig.(b) with tree 903, object A is 
located deep down the tree 903, and hence many merge operations are needed to 
propagate object A's opacity quadtree up towards object B. Finally, in Fig. 9(c) with tree 
905, object A is again located deep down the tree 905. However, object B is also located 
deep down the tree 905. In fact object B is as close to A in the tree 905 as in the tree 901 . 
20 Thus the amount of processing required for the tree 905 is similar to the tree 901 as far as 
the obscurance of object B is concerned. 

The worthiness ratio can be augmented with depth information, as follows 



worthiness = area/(No. nodes x relative depth) (1) 

25 

The term relative depth needs to be quantified when there is more than one 
object being obscured (i.e. objects of high complexity such as images). Preferably the 
depth is taken relative to the nearest object being obscured since this depth represents the 
distance that the opacity information needs to be propagated for some benefit to be 
30 realised. 

5. 1 .2 Calculation of the Worthiness Ratio 

Since there are preferably three different object opacity values, namely fully 
transparent, fully opaque and partially opaque, the 'area' of an object needs to be 
precisely defined when computing the worthiness ratio as described in section 5.1.1. If 
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where a composite object 1000 is worthy if the area of overlap 1001 is less than or equal 
to the area of the largest circle 1003 that can be enclosed within object A; 

(ii) if a worthy, fully opaque rectangular object is placed over another worthy, 
fully opaque object (i.e. not necessarily rectangular), and if the area of overlap between 

5 the objects is defined solely by rectangular edges, then the composite object is also 
worthy. Fig. 11 shows a composite object 1100 where the overlapping area 1101 is 
defined by rectangular edges. Further, a composite object made up of all worthy, fully 
opaque and rectangular objects is also worthy. 

(iii) If a worthy, fully opaque circular object is placed over any other worthy 
10 fully opaque object, then the resulting composite object is also worthy. 

The worthiness ratio, described in section 5.1, can be used to determine on the 
fly whether an object is worthy of optimisation or not. The worthiness ratio described in 
section 5.1 is simple to compute and is accurate. The rules above however, are useful 
since they indicate that obscurance analysis is widely applicable, since a typical image 
15 comprises of mainly over operators. When worthy obscurers are composited together, the 
resultant composite object can in most cases be used as obscurers as well. Thus there are 
few situations where after propagating the opacity information of an object a long 
distance, the resulting object turns out to be unworthy. 

5.2 Processing Small Objects 

20 Many images contain a very large number of small objects, and hence a highly 

efficient method for handling the small objects is essential. Although the use of the 
worthiness ratios discussed in the previous section is capable of masking out small 
objects as poor potential obscurers, the worthiness ratio may not save sufficient 
processing time for an image with many small objects. This is because even though the 

25 objects have been prevented from taking part in the obscurance analysis, their opacity 
quadtrees have already been constructed prior to the computation of their worthiness 
ratios. Constructing a large number of opacity quadtrees can be time consuming. 

Small objects can be afforded special treatment, which in many cases removes 
the need to construct their opacity quadtrees altogether. For example, where only the 

30 inner _op region of an object is required, knowing that the object is small (which is 
deduced from the object's bounding box) allows the object to be treated as if it were 
completely transparent, and hence its opacity quadtree simply consists of a single node. 
Performing a merge between such a quadtree with any other quadtree is a trivial 
operation. 

35 Apart from graphics primitives, certain composited objects can also be deduced 

as small and hence be processed with similar ease. Table 6 below lists the expected size 
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of a composite object as a function of the sizes 
composite, for each type of operator. 



of the individual objects that make up the 



Table 6 
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nodes, where q is the depth of the quadtree, rather than just 1 node as for the inner _op 
case. Fig. 12 shows an example of an opacity quadtree 1200 for a small object. 

For a quadtree of depth 7, which would result in a mesh size of 2.32mm for a 
render image of A4 size, the number of nodes is equal to 4(7) + 1 = 29. This is still a 
5 small figure, at least an order of magnitude smaller than that of a moderate size object. 
Thus processing merge operations on such quadtrees is still relatively efficient. 
Furthermore, the need for the outer _max region of small objects does not occur too often 
since when small objects are present in an image, they are usually composited over other 
objects. 

10 5.3 Incorporating the processing of Small Objects and the Worthiness Ratio 

With regard to the quadtree propagation method outlined in Section 2.5, the 
incorporation of the worthiness ratio and the treatment of small objects lead to the 
following additional sub-steps in the first and second traversals,(ie. steps 4101 and 4103), 
respectively, of the GOB tree. 

15 At step 4101 of Fig. 41, small objects can be identified in a bottom-up manner, at 

the same time as the complex objects are identified. Graphics primitives can preferably 
be marked as being small or large based on their bounding boxes and composite objects 
can preferably be marked based on the markings of the individual objects as dictated by 
Table 6. To facilitate the calculation of the worthiness ratio, a distance value can be 

20 assigned to each node signifying the distance to the nearest complex object appearing at 
or in a subtree below the node. The distance value can be determined in accordance with 
the following rules: 

(i) At a leaf node, if the object is a complex graphics primitive, then assign to it 
a distance value of zero. As at step 4101, the object can be marked as a candidate for 

25 optimisation; and 

(ii) The distance value of a parent node is the minimum of the distances of its 
children, plus 1. Again, as at step 4101, if one of its children is a candidate for 
optimisation, then the node can be marked as being a candidate as well. 

As stated in Section 2.5, the second traversal of the GOB tree, at step 4103, 
30 comprises of two sub-steps: first a top-down procedure for identifying what opacity 
information is needed at each node, followed by the actual computation of the opacity 
values performed in a bottom-up manner. In the top-down procedure, the determination 
of what information is required from a child node based on the information requested by 
the child's parent node is simplified when the parent node is small. If this requested 
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information is inner_op only, then the child node can be ignored, since the inner_op 
region of a small object is a trivial quadtree. 

As for the calculation of the worthiness ratios, the value for the relative depth , as 
discussed above, needs to be calculated for each node as the tree is traversed downward. 
5 The calculation of relative depth can be performed as follows: 

(i) At the root node, the relative depth is the distance value assigned to the node 

in the first traversal, at step 4101 ; 

(ii) each time a branch is traversed downward, the relative depth is incremented 
by one. However, if the value thus obtained is larger than the distance value assigned to 

10 the node being visited, then the relative depth is set to the node's distance value; 

. In the bottom-up procedure of step 4103, as opacity quadtrees are constructed 
and propagated upwards, if a small object is encountered and its required opacity 
information is inner op only, then an opacity quadtree comprising of a single fully 
transparent node that covers the entire render space is constructed. In addition, if the 

15 opacity information required at a node is inner_op only, then the node's worthiness ratio 
can also be determined. If the ratio is below some threshold, then the following steps can 
be taken: 

(i) the opacity quadtree of the object can be replaced by a quadtree composing of 
a single fully transparent node; and 
20 (ii) the object can be labelled as small. 

Since there is a possibility of a relabelling of nodes in regards to their sizes, the changes 
need to be propagated upwards. Thus as the GOB tree is traversed, the size of each object 
can-be redetermined based on the sizes of its children in accordance with Table 6. 
25 It has been shown above that the obscurance analysis can be selectively applied, 

and a heuristic has been described for identifying objects that are worthy candidates for 
optimisation, in the form of the worthiness ratio. The heuristic essentially gives an 
indication of how large an object needs to be in relation to other parameters, before it can 
be considered as a candidate. A natural question that arises is what typically is the size of 

30 such an object. 

Knowing the typical size of an object allows an indication to be determined of 
whether obscurance analysis is worth doing at all. For example, if the size of such objects 
is very large, then it is not worth performing the obscurance analysis since very large 
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objects seldom occur. On the other hand if the size of the objects is very small, then the 
worthiness ratios of objects does not need to be determined. 

The value of the worthiness ratio of an object not only depends on the object's 
properties such as size, but is also dependent on the objects being obscured. Obscurance 
5 analysis is only of benefit when applied to complex graphics primitives such as images. 
These are objects that require considerable computational resources to process, and hence 
the removal of redundant processing results in shorter overall rendering time. Simple 
objects can be more quickly processed and rendered in full, rather than having to perform 
obscurance analysis by propagating quadtrees. 
10 The time needed to process an image is proportional to the area of the image and 

the render resolution. Thus, when an image is obscured by an object, the expected time 
saving can be expressed as 

t/ = k . area . resolution 

15 

where k is some constant, and area is the area of the obscured region. This can be related 
to the area of the obscuring object itself, denoted by area obh by adding a constant 
representing the fraction of the obscuring object that actually overlaps the image, as 
follows 

20 

// = a . k . area 0 bj . resolution (2) 

For processing quadtrees, it is assumed that the time for a single merge operation is 
proportional to the perimeter of the objects. Performing obscurance analysis typically 

25 requires 2 to 4 merge operations at each node in the GOB tree representing the image (i.e. 
1 operation to compute the opacity quadtree of the parent node, and 1 to 3 operations to 
propagate the parent's obscurance quadtree down to the child nodes). Thus if a potential 
obscurer is located N distances away from an image object, then an average of 3N merge 
operations is needed to process the obscurance. The time needed to process the 

30 obscurance is then 

tp — h. perimeter . 3N (3) 
where h is some constant. For obscurance analysis to be beneficial, 
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t, > t P or 

(4) 

a.k. area obj . resolution > h . perimeter . 3N 

5 is required. execution time of an image processing 
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10 processing module). Eq (4) then becomes: 
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where^^^^ 
15 .presentation. For a quadtree of depth 7 and a render ^"J^ ^ 
size 2.32 mm. Also assuming a rendering resoluUon of 400 dpi and t 
object is a square of sides a mm, then: 

area obJ = a mm 2 ; and 
20 perimeter = 4a mm = 4a/2.32 unit cell. 



Substituting into Eq (5), 

. a . (1/28700) . a . (400/72) > 0.082 x 10* . (4/2.32) . 3N. 
25 where a > 0.00219 Ma mm. overlaps ^ 

Finally, assuming that on average, 20 /o of the 



image, the relationship between 



a and N is then tabulated in Table 7. 



Table 7 



N 


a (mm) 


500 


5.48 


1000 


11.0 


2000 


21.9 


5000 


54.8 
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10000 111.0 



It can seen from Table 7 that for an object of reasonable size, say 2cm x 2cm, 
benefits in performing obscurance analysis on the object can be expected, even when the 
object is some 2000 distances away from an image object in the GOB tree. Thus, 
obscurance analysis using quadtrees improves rendering efficiency. 

6.0 Using Obscurance Quadtrees to Limit the Processing when Rendering 
Images 

As described above in the preceeding sections, the resulting product of 
obscurance analysis is a set of obscurance quadtrees which can then be used to avoid 
redundant work in processing and rendering objects or parts of objects that are obscured 
by others. Objects that are simple to render are not assigned an obscurance quadtree, and 
are rendered in a conventional manner. 

For complex objects that have been marked as candidates for optimisation, an 
obscurance quadtree has been assigned to each object. The obscurance quadtree can be 
overlaid on top of the object to identify regions that are hidden and need not be rendered. 

Fig. 43 is a flowchart showing a method of rendering an image using quadtrees 
in order to avoid redundant work in processing and rendering objects or parts of objects 
forming the image. Opacity and obscurance quadtrees have preferably previously been 
propagated for the image using the methods described in section 2.5. The process begins 
at step 4301, the GOB tree representing the image is traversed in order to find a first 
object that has been previously marked as a candidate for optimisation. The object has 
been marked as a candidate for optimisation using the worthiness ratio as discussed 
above. At the next step 4303, if there are no objects marked for optimisation then the 
image is rendered in a conventional manner, at step 4315, and the process concludes. 
Otherwise, the process continues to step 4305, where the object image is partitioned into a 
plurality of equally sized small rectangular image cells. At the next step 4307, the 
obscurance quadtree that has been assigned to the first marked object is overlayed on the 
object. At the next step 4309, the obscurance quadtree is traversed to identify which of the 
image cells for the object fall under the visible region of the quadtree. At the next step 
4311, the object is rendered only in regions that fall under the visible region of the 
quadtrees associated with the object. At the next step 4313, if the GOB tree representing 
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quadtree. Each 'pixel' in the map corresponds to a single image cell. The map is then 
used to discriminate between visible and hidden image cells, at step 4313, whereby the 
pixels act as flags indicating visible and hidden image cells. The advantage of the 
quadtree centric method is the fact that the obscurance quadtree for an object is traversed 
5 only once. One disadvantage is the creation and storage of the intermediate 'visibility' 
maps. However, by a careful choice of the order in which a quadtree is traversed, and by 
using a suitable format, the map can be created efficiently and without excessive storage 
requirement, as will be described in more detail below. 
6.2. 1 Preferred Format of the Visibility Map 

10 Since there is a 1-to-l correspondence between a pixel in the visibility map and 

an image cell, the size of the map can be determined by the dimensions of the image in 
terms of the number of cells. For large images, representing the map in the raw pixel 
format can consume substantial storage space. Preferably, a form of run-length encoding 
is utilised. A visibility map can be decomposed into individual rows where on each row, 

15 consecutive runs of visible cells are represented by a pair of numbers signifying the 
starting and ending (horizontal) positions of the sequence. Although it is possible that 
run-length encoding can take up more storage space than the raw pixel format, for 
example, if there is a large number of small isolated visible cells, in practice this rarely 
occurs. 

20 6.2.2 Order of traversal of the obscurance quadtree 

One advantage of the run-length format is that it allows the visibility map to be 
generated more efficiently. The visibility map can be generated by traversing the 
obscurance quadtree and appending visible cells to the map when visible nodes are 
encountered. Appending extra cells under the run length encoded format typically 

25 requires the update or insertion of two numbers, as opposed to updating the values of 
individual pixels in the raw pixel format. However, some search can be necessary to 
locate the appropriate numbers or insertion point if the new cells are added to the middle 
of a row. Fortunately, by traversing the quadtree in a certain order, described in detail 
below, for the appropriate numbers or insertion point is not necessary. New cells can 

30 always be appended to the end of each row, which can be performed without any search. 

The order in which the obscurance quadtree for an object is traversed depends on 
how the obscurance quadtree is oriented with respect to the object's image coordinate 
space, due to the presence of rotational and reflection transformations. Regardless of how 
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concludes. Otherwise, the process proceeds to the next node at step 1313, and then 
returns to step 1315. 

Accordingly, by beginning with a map in which all cells are hidden, and then 
traversing the quadtree and 'coloring' in cells lying underneath 'visible' quadrants of the 
5 quadtree, cells that lie outside of the span of the quadtree can automatically be labelled as 
'hidden' since they are never visited. 

Step 1309 is the most difficult step of the method as it is preferably achieved by 
scan converting the rotated quadtree in the coordinate space of the visibility map. The key 
issues involved in performing step 1309 will now be examined in more detail. 
10 6,3 Mapping Visible Cells 

Having overlaid a rotated obscurance quadtree on top of an image object, as at 
step 4307, a visibility map can be generated for the image, at step 4309, by identifying 
which image cells fall under the visible region of the quadtree. Cells that partially fall 
under the visible region are also considered to be visible. Preferably the visible quadrants 
15 of the obscurance quadtree are scan converted by converting the quadtree into a series of 
(i.e. possibly complex) polygons and then scan converting the polygons. Alternatively, 
the individual visible quadrants can be scan converted separately. 

One advantage of scan converting the individual visible quadrants is that it does 
not require the intermediate conversion step, and that scan converting a 
20 rectangle/parallelogram is considerably simpler than scan converting a complex polygon. 
One disadvantage of scan converting the individual visible quadrants is that the number of 
visible quadrants in the quadtree can be large and hence scan converting them 
individually can be time consuming. In particular, edges that are common between 
adjacent visible quadrants need to be scan converted twice, whereas by converting the 
25 quadtree into a series of polygons and then scan converting the polygons, such edges do 
not exist. 

Preferably, adjacent quadrants 2301, 2303 and 2309 can be merged 2307 
together to eliminate their common edges 2305, as illustrated in Fig. 23. Preferably, this 
is only performed on a selective basis to minimise processing time. In addition, the 
30 resulting polygons are preferably simple so that they can be scan converted easily. The 
resulting polygons can be restricted to rectangles. 

The packed data representation described above can be utilised to efficiently 
merge quadrants lying on levels 0 and 1 that share a common ancestor into larger 
rectangles. In other words, each quadtree can be divided into blocks of size 4 by 4 cells 
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and adjacent quadrants within each block can be merged together. Preferably, no cross- 
block merging is performed. The object of the merge operations is to describe the visible 
region in each block by a set of non-overlapping rectangles. For example, Fig. 24 shows 
an irregularly shaped visible region 2411 that has been converted into a series of 
rectangles 2403, 2405, 2407 and 2409. The visible region 2411 shown in Fig. 24 is 
comprised of 1 1 cells (i.e. not including the grey cells (e.g. 2401)) and can be represented 
by the four rectangles 2403, 2405, 2407 and 2409. 

For maximum effectiveness, the number of rectangles is preferably kept to a 
minimum. Given an arbitrary region, determining a minimum set of rectangles that 
precisely represent the region is generally a complicated exercise. However, if the 
merging of adjacent quadrants is limited to within each 4 by 4 block as described above, 
then determining a minimum set of rectangles that precisely represent the region can be 

implemented using lookup tables. 

As described above, in the packed data representation, the values of sibling 
nodes on the bottom most level can be packed into a single number. For obscurance 
quadtrees where each node can take on one of two values, 4 bits are needed (as opposed 
to 8 bits for opacity quadtrees). A 4 by 4 block thus requires a total of 16 bits. Fig. 14 is 
a flowchart showing a 'rectangularisation' method for fitting rectangles within a 4 by 4 
block using table look-up. The process begins at step 1401, where the packed data 
structures of the four 2 x 2 sub-blocks are combined into a single 16 bit word. At the next 
step 1403, the 16 bit word is used as an index into a look up table. Each entry in the table 
holds the positions of the left, right, top and bottom sides of a rectangle that can fit into 
the region denoted by the look up word. Since each side of the rectangle can take on one 
of four possible positions within a 4 by 4 block, each side of the rectangle can be 
represented by a 2-bit number. Thus a single byte of data is sufficient to represent the 
four sides of the rectangle, resulting in the look up table being of size 216 = 65536 bytes. 
At the next step 1405, the rectangular region returned by the table look up constitutes one 
of the resulting rectangles. At the next step 1407, the rectangle just found is removed 
from the visible region and a new 16-bit word representing what visible region that 
remains is constructed. The process then returns to step 1403, after checking whether or 
not all of the visible regions have been represented at step 1409. 

Fig. 25 shows the preferred format of a look-up table 2501 used in step 1403. 
White (e.g. 2503) and solid grey (e.g. 2505) cells denote visible and hidden quadrants 
respectively. Hashed cells (e.g. 2507) denote visible cells that are to be merged into a 
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single rectangular region. The top block 2509 shows how the packed data contents of the 
4 sub-blocks (e.g. 2513) can be combined to give a 16-bit index for the look up table 
2501. The bottom block 2511 shows how the 8-bit entry in the table 2501 is broken up 
into coordinates of the 4 sides of the rectangular region (i.e. denoted by the hashed cells) 
5 residing within the visible region. 

The look up table 2501 is preferably set up so that each entry in the table 
represents a rectangle that makes up a set of minimum number of rectangles that fits into 
the region denoted by the index to that entry. If each entry in the table 2501 represents a 
rectangle that makes up a set of minimum number of rectangles that fits into the region 
10 denoted by the index to that entry, then the method of Fig. 14 will return a minimum set 
of rectangles. 

6.3.1 Effectiveness of the 'rectangularisation' method 

An indication of the benefit of the rectangularisation method of Fig. 14 can be 
obtained by comparing the total perimeter of the resulting rectangles with the total 

15 perimeter of the individual quadrants. The perimeter of an object gives a good estimate 
of the processing time needed to scan convert the object since scan conversion operates 
on the object's edges. The inventor found through experimental results obtained from 
fitting rectangles to randomly generated 4x4 blocks an improvement of 20%. 

The 20% figure is due to the fact that randomly generated blocks tend to be poor 

20 candidates rectangularisation. As a result, the number of cells able to be merged tends to 
be low. The actual benfit in practice is higher since real quadtrees have more uniform 
regions, which make the quadtrees better suited for rectangularisation. 

6.4 Optimal scan conversion ordering 

25 In Section 6.2.2, a method of selecting an optimal order for traversing an 

obscurance quadtree was described. The order for traversing the obscurance quadtree can 
be selected regardless of the degree of rotation and shearing, such that the visibility map 
can be built up in a highly efficient manner. Allowing new visible cells to be appended to 
the end of each row rather than in the middle, thereby eliminating the need to search for 

30 the insertion point. Such a method is possible since the obscurance quadtree is mapped 
(ie. scan converted) onto the visibility map on a quadrant-by-quadrant basis. The order in 
which the quadtree is traversed guarantees that each quadrant is scan converted after its 
neighbour on the left. 
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A method of utilising the above properties for determining the priority order of a 
set of non-overlapping rectangles on a 4 x 4 block is described below. The method works 
by examining the block on a quadrant by quadrant basis. Whilst doing so, the method 
maintains a working priority list by inserting rectangles that are encountered into the list 
5 at appropriate locations. 

Fig. 15 is a flowchart showing a method for determining the scan conversion 
priority order of a set of non-overlapping rectangles on a 4 x 4 block. The process begins 
at step 1501, where the quadrant numbered 1 (left most) of the block is examined. 
Properties (v) and (vi) are utilised to determine the relative priorities of all rectangles that 

10 appear in quadrant 1. References are added for the rectangles of quadrant 1 to the start of 
a working priority list in the order of the relative priority, followed by a dummy reference 
as a marker for later use. At the next step 1503, quadrant numbered 4 (right most) is 
examined. Again properties (v) and (vi) are utilised to determine the relative priorities of 
all rectangles that appear in quadrant 4. References for these rectangles are added to the 

15 end of the working priority list in the order of their relative priorities. It can be seen that 
if one of the rectangles (there is at most one) of quadrant 4 has already been encountered 
in quadrant 1 then the rectangle must occupy the right most cell of that quadrant, and is 
hence the lowest priority rectangle identified in step 1501. In addition, the rectangle must 
also occupy the left most cell of quadrant 4, and hence has highest priority out of the 

20 current set of rectangles in quadrant 4. Thus, the current set of rectangles can be joined to 
the end of the rectangles obtained in step 1501 to form a larger priority list. The dummy 
reference marker inserted at the end of step 1501 can also be removed. The process 
continues at step 1505, where for quadrants (2) and (3) (it does not matter which of these 
two quadrants is processed first), the same procedure of utilising properties (v) and (vi) is 

25 used to determine the relative priorities of the rectangles in quadrants 2 and 3. How, the 
rectangles of quadrant 2 and 3 are inserted into the working priority list depends on 
whether one of the rectangles has already been encountered in quadrant 1 and 4. If none 
of the rectangles of quadrant 2 and 3 have been encountered in quadrant 1 and/or 4 then 
property (iv) can be used to insert the new rectangles where the dummy reference marker 

30 was placed in step 1501. In this instance, the dummy marker could not have been 
removed at the end of step 1503 since if there exists a rectangle that resides on both 
quadrants 1 and 4, then the rectangle must also reside in quadrants 2 and 3 (i.e. the 
rectangle must occupy the four centre cells), and thus one of the rectangles in the current 
set must have been encountered previously. If one of the rectangles of quadrant 2 and 3 
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scan conversion are run length-like coordinate pairs representing the starts and ends of 
pixel runs. The run length-like coordinate pairs would then need to be written pixel by 
pixel onto a visibility map and depend on the size of the map. A larger map would mean 
there are more pixels to 'turn on\ Another disadvantage of using raw bitmap format is 
5 the larger storage memory needed to accommodate the visibility map. Although this is 
probably not an issue for a single image since the resolution of the map is rather coarse 
compared to that of the image, it could become significant if there are many images 
present. 

Another alternative method for generating visibility maps is to retain the use of 
10 run length encoding but avoiding the determination of priority ordering by pre-computing 
and storing all the necessary data in a look up table. Each entry in the look up table needs 
to hold the coordinates of up to 8 rectangles that make up each unique 4 by 4 block, rather 
than just a single rectangle as described in Section 6.3. In addition, the look-up table 
would need to include extra data to allow the priorities of the rectangles to be determined 
15 under any arbitrary orientation. It can be shown that such a look up table needs to be of 2 
Megabytes in size, which is rather excessive. 

7.0 Combining Obscurance Analysis and GOB tree Optimisation 

Of the different binary operators, the over (or rover) and in (or rin) operators are 
20 associative. That is, the composition a over (b over c) is equivalent to (a over b) over c. 
This property allows a sequence of over and rover operations, or a sequence of in and rin 
operations, to be rearranged and represented by a right leaning GOB tree. The 
rearrangement is known as GOB tree optimisation. A right leaning GOB tree is desirable 
since it can be rendered more efficiently. 
25 This section describes how obscurance analysis can be combined with GOB tree 

optimisation. Preferably, the obscurance analysis and GOB tree optimisation methods are 
performed in parallel in order to reduce the total number of GOB tree traversals. 

7.1 GOB Tree Optimisation 

A GOB tree can be optimised by firstly, re-configuring the GOB into a right 
30 leaning structure where sequences of over and rover, or in and rin are encountered. 
Secondly, nodes can be arranged in such a structure so that more complex objects are 
located towards the bottom. 

For example, consider the GOB tree 3400 of Fig. 34. B is the most complex 
object in the chain. By converting some operators to rover , B can be placed at the bottom 
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of the chain of GOB tree 3401. This conversion can be performed by keeping track of the 
most complex object found on the left side as the tree 3400 is traversed from the top 
down, and then reverse the chain from there onwards upon reaching the end of the chain. 
Thus in the example of Fig. 34, the chain of nodes found below B (i.e. B-C-D-E) in tree 
3400, is reversed to form E-D-C-B in tree 3401. 

The GOB tree optimisation method outlined above generally requires two 
traversal of a chain and performs two updates to the GOB tree. However, the first 
traversal can be eliminated. For example, consider the GOB tree 3500 of Fig. 35 that is 
logically equivalent to the tree 3400 of the example in the preceding section. The GOB 
3500 can be the original tree prior to having been transformed into a right leaning 
configuration. Assuming that object B of the GOB 3500 has been previously identified 
and tagged as the most complex object. Further assuming that B's ancestors have also 
been tagged as shown in Fig. 35. The tagging can be done at the same time as the 
computation of the objects' complexities. 

Fig. 16 is a flowchart showing an alternative method of optimising a GOB tree 
using obscurance analysis. The process begins at step 1601, at the top node of the GOB 
tree 3500. The GOB 3500 is traversed in a top-down manner. At the next step 1603, the 
child node with a tag (e.g. 3501) is identified. The process continues at the next step 
1615, where if the tagged node 3501 is on the left child, then the process proceeds to step 
1617 before proceeding to step 1605. At step 1617, the node's 3501 children are swapped 
and the operator is replaced by itsopposite, as shown by the GOB 3601 of Fig. 36(a). At 
step 1605, if the left child is a leaf node, then move down to the right child node, at step 
1607, and return to step 1601. Otherwise the left child is denoted by the expression (A op 
B),,at step 1609. Also the right child is set to C. Thus, the subtree below the current 
node is denoted by (AopB)op C. At the next step 1611, the subtree is rearranged 
according to the following transformation: 
(A op B) op C = A op (B op C) 
Before the transformation can be applied however, if the operator at the left child 
is not the same as that at the parent node, then the former needs to be inverted, as shown 
by the tree 3605 shown at Fig. 36(b). The resulting GOB tree 3603 after the 
transformation is shown in Fig. 36(c). If the whole tree 3500 has not been traversed at 
step 1613, then the process returns to step 1603. Otherwise the process concludes. 
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The resulting GOB tree 3500 at the completion of the method of Fig. 16 is 
shown as the GOB 3700 in Fig. 37. Note that the GOB 3700 has a different structure but 
is otherwise logically equivalent to the tree 3400 shown in Fig. 34. 

7.2 Performing obscurance and GOB tree optimisation using a minimum 
5 number of tree traversals 

The number of GOB tree traversals necessary to separately perform obscurance 
analysis and GOB tree optimisation can be summarised as follows. For obscurance 
analysis, one traversal is required to identify small objects and analysis candidates, one 
traversal is required to compute the opacity quadtrees and one traversal is required to 

10 compute the obscurance quadtrees. For GOB tree optimisation, one traversal is required 
to compute the bounding boxes (and complexity metrics) and one traversal is required to 
reorganise the GOB tree into a right-leaning structure. Thus up to 5 traversals of the 
GOB tree can be required in total. Preferably, the obscurance analysis and GOB tree 
optimisation method can be performed simultaneously. Fortunately, the computation of 

15 bounding boxes is quite independent of the obscurance analysis process and can hence be 
done in parallel with the first traversal of the quadtree propagation, as at step 4101. Note 
however, that bounding box minimisation in the presence of in and out binary operators 
can lead to part of the GOB tree being retraversed multiple times, and possibly can result 
in parts of the GOB tree being removed due to total obscurance. If retraversal occurs, the 

20 small object and optimisation candidate flags and the "distance to the nearest complex 
object" value at each node for use in obscurance analysis need to be redetermined with 
each retraversal. 

Combining the second traversal in the GOB tree optimisation process with the 
remaining steps in obscurance analysis is more complicated, since GOB tree analysis can 

25 alter the structure of the GOB tree, and hence can invalidate attributes assigned to each 
node for use in obscurance analysis. To avoid this problem, the restructuring of the GOB 
tree is preferably performed during the third traversal of the obscurance analysis method 
as at step 4107, when the final obscurance quadtrees have been obtained for nodes 
undergoing the transformation. 

30 An example in which oscurance quadtrees and GOB tree optimisation is carried 

out in one traversal is given in section 8.3.1. 

7.2.1 Computing Obscurance Quadtrees and GOB Tree Optimisation in One Traversal 
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Fig. 17 is a flowchart showing a method of computing the obscurance quadtrees 
and reorganising the GOB tree into a right leaning structure in a single traversal of the 
GOB tree. The method of Fig. 17 is dependent on the following assumptions: 

(i) assume the second traversal of the GOB tree has been completed as at step 
4105, and hence the opacity quadtrees of each node in the GOB tree have been 
determined. At the same time the bounding boxes and the complexity metrics, which is 
dependent on the dimensions of the bounding boxes, have been determined for each node; 
and 

(ii) assume that the most complex objects within sequences of over/rover have 
been identified and marked as described in Section 9.2. How this tagging can be done 
within the same GOB tree traversal as the computation of the bounding boxes will be 
described in more detail later in the document. 

As described above the computation of the obscurance quadtrees is a top-down 
procedure where the obscurance quadtree of a parent node is propagated down and 
combined with the opacity quadtrees of its children to form their obscurance quadtrees. 
Once the obscurance quadtrees of the child nodes have been determined, their opacity 
quadtrees and the obscurance quadtree of the parent node are no longer needed and can be 
discarded. Thus the child nodes can be rearranged without affecting the validity of the 
opacity information of their descendants. 

Consider the example shown in Fig. 35, and the sequence of GOB tree 
transformations that result in a right leaning structure shown in Figs. 36(a)-(c) and Fig. 
37. Step 1601, as shown in Fig. 36(a), involves a simple operand swap and operator 
inversion from over to rover and hence can be done safely without affecting the opacity 
information of any node. Step 1607, as shown in Fig. 36(b), is also a simple operand 
swap. Step 1609 however, is a rearrangement of the operator sequence using the 
associative property of over. The transformation is illustrated again in Figure 38. 

Notice that node F' in the transformed GOB tree 3603 does not represent the 
same entity as node F in the original GOB tree 3500, since F = E rover D whereas F* = D 
rover G. Thus, the opacity quadtree determined for F cannot be used as if it were the 
opacity quadtree for F'. Preferably, the opacity quadtree for F' is redetermined from the 
opacity quadtrees of D and G, and the process can then proceed normally. Alternatively, 
the extra computation can be discarded altogether. In this connection, the structure of the 
subtrees below G, E and D (E and D may represent complex subtrees rather than the 
primitive nodes shown in Fig. 38) do not change in the transformation. Once the 
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obscurance quadtrees for G, E and D have been determined, the opacity and obscurance 
quadtrees of their ancestor nodes can be discarded. 

With reference to Fig. 38, the process of Fig. 17 begins at step 1701 at the top 
node of the GOB tree 3500. The GOB 3500 is traversed in a top-down manner. At the 
5 next step 1703, a child node with a tag (e.g. 3501) is identified. If the tagged node 3501 
is on the left child, then the children are swapped and the operator is replaced by its 
opposite. At the next step 1705, if the left child is a leaf node, then the traversal moves 
down to the right child node, at step 1707, and returns to step 1703. Otherwise, the left 
child is denoted by the expression (A op B), at step 1707. Also, the right child is denoted 

10 by C. Thus the subtree below the current node is denoted by (A op B) op C. At the next 
step 1711, the obscurance quadtrees for nodes F and G, and subsequently for nodes E and 
D are determined. At the next step 1713, the subtree is rearranged according to the 
following transformation: 

(AopB)opC =Aop(BopC) 

15 The obscurance quadtree for F' is now incorrect and can be discarded. The 

correct obscurance quadtree for F'needs not be determined since the obscurance quadtrees 
of its children, D and G, have already been obtained. F' is then tagged as having already 
been processed, to avoid having to propagate down to D and G its incorrect obscurance 
quadtree. If the whole tree has not been traversed at step 1715, then the process returns to 

20 step 1703. Since E may be a complex subtree rather than a primitive node, it may contain 
rover/over operators which need to be transformed over to the right side of the tree in the 
same manner as already discussed. When E has been fully traversed, processing returns 
to the right hand side. Subsequently, node F'is skipped over since it has already been 
processed. 

25 7.2.2 Computation of bounding boxes for GOB tree optimisation 

The computation of bounding boxes for all nodes in the GOB tree is necessary 
since GOB tree optimisation makes use of complexity metrics that are derived from the 
dimensions of the bounding boxes. Bounding boxes are also used during the actual 
compositing and rendering phase. Preferably bounding box minimisation remains 

30 unchanged except that it is now performed in parallel with the first traversal of the 
obscurance analysis. 

7.2.3 Tagging the most complex node for GOB tree optimisation 

Since the complexity metrics used in GOB tree optimisation rely on the 
dimensions of the bounding boxes, complexity metrics can be determined once the 
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bounding boxes have been determined. From the complexity values, the most complex 
object can be identified and the necessary tags made to the GOB tree to facilitate its 
optimisation. An extra GOB tree traversal can be avoided by performing the tagging in 
the first traversal where the computation of bounding boxes is finalised. 

5 In the post-order traversal stage at step 4105, at each parent node, the bounding 

boxes of its children and hence their complexity values can be fully determined. The 
complexity values can be compared, and a tag placed on the child node with the highest 
value. This can be repeated as the GOB tree is traversed upward. The result of the 
tagging is that at each node, there is a sequence of tags that leads to the most complex 

10 object in the subtree below it. Not all tags can be subsequently used in the GOB tree 
optimisation process. Only those that lead to the most complex objects in sequences of 
over/rover, or in/rin operators can be used, the remaining nodes are ignored. 
8.0 Mimimisimig Memory Usage 

When operating under an environment of limited memory resources, the method 

15 for propagating quadtrees can be modified to minimise the memory footprint of the 
method. 

The major consumer of memory within the preferred method for propagating 
quadtrees lies in the storage of opacity quadtrees generated during the second GOB tree 
traversal, as at step 4103. At the completion of this traversal, there can potentially be one 

20 opacity quadtree stored at each node in the GOB tree. Comparatively, the storage 
requirement for obscurance quadtrees is considerably less, since they are only kept at leaf 
nodes and only for complex primitives (eg. images). The obscurance quadtrees generated 
for non-leaf nodes during the third GOB tree traversal, at step 4107, are immediately 
discarded once they have been propagated downwards. It should be noted that the opacity 

25 and obscurance quadtrees of the same node are not stored simultaneously. Once the 
obscurance quadtree has been determined for a node, its opacity quadtree can be 
discarded. 

8.1 Discarding UmwaiULted Opacity Quadtrees 

All opacity quadtrees need to be maintained through to the third traversal at step 
30 4107. However, as discussed above only relevant opacity quadtrees are generated 
through the treatment of partial opacity information. The opacity quadtree of a node is 
determined for two reasons. Firstly, to compute the opacity quadtree of its parent node, 
and secondly, if the parent is a binary operator, to compute the obscurance quadtrees of 
itself and its sibling. The second reason for storing opacity quadtrees through to the third 
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traversal, at step 4107, can be avoided if the parent node is a unary operator or if the 
parent node is a binary operator but the opacity quadtrees of one or both of the siblings do 
not take part in the computation of the siblings obscurance quadtrees, as dictated in Table 
2, reproduced below, as Table 8„ for convenience: 
5 Table 8 



Operator 


Left operand 


Right operand 


Over 

Out 

In 

Atop 
Xor 

plusc, plusw, rops 


Qp 

Q P u{Q R ='o'} 
Q P u{Q R = '-'} 

Qp^{Qr='-'} 
Qp«-'{Ql='o' andQ R = 'o'} 
Qp 


QpU{Q l = 'o'} 
Qp^{Q R = 'o'orQ L = '-'} 
Qp^{Q R = 'o'orQ L ='-'} 
Q P u{Q L = 'o'andQ R ='o'} 
Qp ^{Ql = *o' and Q R = 'o'} 
Qp 



Depending on whether the sibling nodes have been tagged as candidates for 
optimisation, Table 8 can indicate whether the opacity quadtrees of either, both, or none 
of the siblings are needed. Opacity quadtrees that are not needed can be discarded as 

10 soon as the opacity quadtree of the parent node has been determined. For example, if the 
right child of an over operator is a candidate for optimisation, then only the opacity 
quadtree of the left child needs to be kept, since Q R does not appear in the "Right 
operand" column in Table 8 for over. On the other hand, if the left child is a candidate for 
optimisation (but not the right), then the opacity quadtrees for both child nodes can be 

15 discarded, since neither Q L nor Q R appear in the "Left operand" column for over, 
8.2 Early Computation of Obscurance Quadtrees 

Memory usage can also be reduced by beginning a computation of obscurance 
quadtrees during the second, as at step 4103, rather than the third GOB tree traversal, as at 
step 4107, so that all opacity quadtrees can be discarded. The computation of obscurance 

20 quadtrees can not be completed until the third traversal however, so the partial results (in 
the form of quadtrees) need to be kept instead of the original opacity quadtrees. For cases 
where only one operand of a binary operator is a candidate for optimisation, this can lead 
to fewer quadtrees being kept. As shown in Fig. 40, B is a candidate for optimisation, 
whereas A is not. From Table 8, B's obscurance quadtree is given by Q P ^{Qr = V or 

25 Q L = *-*}. Thus, if the computation of B's obscurance quadtree is delayed until a third 
traversal of the GOB tree, then the opacity quadtrees of both A and B need to be kept 
until then. However, if B's partial obscurance quadtree is pre-determined, as Qb* = {Qr = 
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'o' or Q L = '-'}, then both Q L and Q R can be immediately discarded after the parent's 
opacity quadtree has been determined. Thus, only one quadtree, Q B \ needs to be stored 
as compared to two. The computation of B's obscurance quadtree can be completed 
during the third traversal, as at step 407, when Q P is known. 

Note that pre-computing obscurance quadtrees can lead to longer computation time 
in cases where both child operands of a binary operator are candidates for optimisation. 
For example, this is true for the out operators. Without pre-computation, it takes two 
quadtree merge operations to compute the obscurance quadtrees of both child operands 
(i.e. first compute right's obscurance quadtree = Q P o{Q R = 'o'}, then left's obscurance 
quadtree = right's obscurance quadtree u{Ql = '-'})• With pre-computation however, it 
takes three merge operations (first pre-compute Q temp = {Qr = '°' or Ql = '-')» tnen 
compute right's obscurance quadtree = Q P o{Q R = 'o'}, and left's obscurance quadtree = 

Qp u Qtemp in the third traversal). 

In a memory constrained system, any slight inefficiency can be more than offset 
by the reduced memory usage. Preferably, pre-computation of obscurance quadtrees can 
be applied selectively (ie. to binary operators with only one child that is a candidate for 
optimisation), rather than to the whole GOB tree representing an image. 
9.0 Compositing Analysis 

Compositing analysis can be used to optimise the time needed to render an object 
by compositing the object only within the object's non-empty, visible region described by 
a quadtree, rather than compositing within the object's bounding box, as has been 
conventionally done. In order to perform this optimisation on a given object, a quadtree 
describing regions where the object is empty and regions where the object is obscured by 
other objects is determined. The quadtree can be determined from the object's opacity 
and obscurance quadtrees. The obscurance quadtree alone does not contain sufficient 
information, since it only describes the obscured regions of an obj ect. 

Compositing analysis can not only identify where an object is empty and need 
not be rendered, but also where the object can be rendered using a simpler and faster load 
rather than a full compositing operation. This functionality requires a new quadtree, 
which is referred to as a compositing quadtree. The compositing quadtree carries three 
distinct node values, as opposed to the two values carried by the obscurance quadtree. 
The three node values are denoted by T and «c\ representing regions where the 
object needs to be rendered, where the object can be rendered by a load and where the 
object must be rendered using a full compositing operation, respectively. With the 
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addition of the compositing quadtree, the obscurance quadtree is no longer required for 
rendering the image. Opacity quadtrees are still required since they are the source from 
which compositing quadtrees are obtained. 

When performing a compositing operation, only the compositing quadtree of the 
5 object being composited is needed. The compositing quadtree of the object being 
composited upon is not required. Preferably, objects that are composited onto others are 
primitives that appear as the left operands of a binary operator in the GOB tree, and non- 
primitives appear on the right. The distinction between primitive and non-primitive 
operands is advantageous when performing PUSH and POP operations when rendering 

10 non-primitive objects. For example, a simple GOB tree 4400 is shown in Fig. 44(a) along 
with the GOB tree's associated rendering instruction sequence 4401. Since the left 
operand A is primitive, the binary compositing operator over is realised by first rendering 
the right operand B and then compositing the left operand A onto the right operand B 
using the instruction "over A". However, when the left operand of a tree 4403 is 

15 complex, as shown in Fig. 44(b), then the opposite happens. As shown by the instruction 
set 4405, instructions are first generated for rendering the right operand B. The result is 
then preferably PUSHed onto a stack and an accumulator is cleared, after which the left 
operand A is rendered. Finally, the over operation is performed by POPing the saved 
right operand B and compositing it onto the left operand using the reverse operator "rover 

20 pop". 

The above-mentioned method for processing operations involving complex left 
operands can lead to substantial amounts of work in computing the necessary information 
for compositing analysis. As will be discussed in further detail later in this document, 
compositing with the right operand of an expression tree when the left operand is 
25 complex does not allow for small objects to be handled efficiently. This problem can be 
overcome by instead of PUSHing and POPing the right operand as depicted in Fig. 44(b), 
preferably the left operand is PUSHed and POPed, as illustrated by the following 
instruction sequence for the example of Fig. 44(b): 



30 



35 



push 
load B 
over pop 
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10.0 Propagating Compositing Quadtrees 

Like the obscurance quadtrees discussed above, compositing quadtrees are 
propagated during a second phase of obscurance/compositing analysis, and are 
propagated downwards from the top of a GOB tree. 

When propagating a compositing quadtree through a binary operator, the 
compositing quadtrees of the child operands are obtained from their opacity quadtrees and 
also the compositing quadtree of the parent node, in a two step process. Firstly, 
obscurance information is extracted from the parent's compositing quadtree and applied 
to the opacity quadtrees of both operands to propagate the obscured region of the parent 
to the operands, which will be described in more detail in Section 11.4. This has the 
effect of converting the operands' opacity quadtrees that lie in the obscured regions into 
fully transparent nodes. Second, the modified opacity quadtrees of the child nodes are 
merged with each other to generate the children's compositing quadtrees. The second 
step depends on the type of binary operator involved. The resulting compositing quadtree 
for the left operand is described in Table 9 below: 
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Table 9 



Operator 


L 


R 


Resultan 








t 


Over 


- 


X 






o 


X 


1 




oc 




1 




a 


a 


c 




OC 


0 


c 


Rover, out 




X 






X 


o 






a 




1 




a 


a 


c 




o 


- 


1 




o 


a 


c 


In 


- 


X 


- 




X 


- 


- 




a 


a 


c 




a 


0 


1 

1 




o 


a 


c 




o 


o 


1 


rin, rout 


- 


X 


- 




X 








o 


X 






a 


a 


c 




a 


o 


c 


Atop 




X 


- 




X 








a 


a 


c 




a 


o 


c 




o 


a 


c 




o 


o 


1 


Ratop, xor 


- 


X 


- 


a 


- 


1 




oc 


oc 


r* 
V* 




a 


o 


c 




o 


- 


1 




o 


a 


c 




o 


o 


- 


Plusc, 




X 




plusw, 


a 




1 


a 


a 


c 


rops 


a 


o 


c 




o 




1 




0 


a 


c 




0 


o 


c 



I:\501203.doc 



59- 



10 



15 



The compositing quadtree for the right operand can be obtained from Table 9 by 
looking up the corresponding entries for the reverse operator. 
10.1 Partial Opacity Information 

As discussed above, for obscurance analysis, processing time can be reduced by 
generating only partial opacity information, where the opacity information is needed. 
This is effective since common operators such as over only require partial opacity 
information (i.e. typically the inner _op region) to generate the necessary obscurance 
information. Unfortunately with the introduction of compositing analysis, it is no longer 
adequate to compute partial opacity information in most situations, as seen from Table 10 
below. Table 10 lists the opacity information required to compute the compositing 
quadtree of the left operand for each binary operator. Table 10 can be derived from Table 
9. It can be seen from Table 10 that in all cases where full opacity information is not 
required, the only information that can be left out is the inner _op region of operands. 

Table 10 



20 



Operator 


L 


R 


over, rin, rout 
rover, in, out 
atop, ratop, xor 
plusc, plusw, 
rops 


both 
outer jnax 

both 
outer jnax 


outer jnax 

both 

both 
outer jnax 



In addition to Table 10, the opacity information needed from the operands to 
compute the opacity quadtree of the parent node, is shown in Table 11. For example, in 
order to determine the inner _op region of the over operator, the inner _op regions of both 
child operands need to be determined. Further, the opacity information needed to 
compute the left operand's compositing quadtree is required. Therefore, the full opacity 
information from both operands is required and there would be no benefit in incorporating 
the treatment of partial opacity information into compositing analaysis. 



25 
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Tablell 



Operator 


Parent 




K 


over, in, 


inner _op 


inner _pp 


inner _op 


plusc, rops 


outer max 


outer max 


outer max 


out 


inner _op 


inner op 


lsl4ltZf frltlA, 




outer max 


outer max 


inner op 


atop 


inner_op 




inner op 




outer max 




outer max 


xor 


inner_op 


both 


both 




outer max 


both 


both 


plusw 


inner_op 


both 


both 




outer max 


outer max 


outer max 



11.0 Processing Small Objects 

The result of not being able to take advantage of partial opacity information is 
5 that an effective means of handling small objects is lost. In some cases, GOB trees 
containing a large number of small objects can be encountered. It would be very time 
consuming to have to compute full opacity and compositing quadtrees for every single 
object of such a GOB. 

Without the use of partial opacity information, another method for handling 
10 small objects is needed. A new method of propagating opacity quadtrees, is described 
below, in which bounding boxes are maintained instead of quadtrees for small objects 
whenever possible (the rectangular regions defined by the bounding boxes are considered 
to be partially transparent). Since bounding boxes are much easier to process than 
quadtrees, it should be possible to handle large numbers of such objects without 
15 consuming significant computing resources. 

11.1 Using Bounding Boxes to represent opacity information for small objects. 
The principle difficulty of using bounding boxes to represent opacity information 

for small objects is the issue of how the bounding boxes can coexist with opacity 
quadtrees in the same GOB tree. A GOB tree usually contains large objects as well as 

20 smaller ones, whose opacity information is best represented by quadtrees. When small 
objects are composited together, the fast computation can be utilised to represent the 
opacity information of the resulting composite object using a bounding box. When a 
large object and a small object are composited together however, a quadtree is preferably 
used except for cases where the composite object is known to be small. 

25 Using bouding boxes and quadtrees in the two instances discussed in the 

previous paragraph works well when the small objects are clustered together in a 
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separable par, of ft. GOB tree. When mis is no. .he case, the use of bounding boxes can 
prove ineffective. For example, a GOB tree 4500 is shown in Fig. 45 where a large 
number of small objects, 4501-4505, are placed over a large object - A'. The composite 
object 4509 is also shown in Fig. 45. The five small objects, 4501-4505 are located 
together and fully reside in a subtree 4507 of their own Chat can be separated out from the 
large object A. Thus a simpler bounding box computation can be performed wthm the 
subtree 4507. At node N, where the small objects, 4501-4505, are composited over A, a 
merge operation between the small objects' overall bounding box and A's opacty 
quadtree can be performed to compute the opacity quadtree for the whole composrte 
object The merge operation can be done with approximately the same efficency as a 

merge between two quadtrees. 

In the example of Fig. 45, the presence of a large number of small objects 4501 - 
4505 can be handled efficiently using bounding box optimisation. 

Now consider the equivalent GOB tree 4600 shown in Fig. 46. In this example, 
; small objects 4601-4605 do not appear by themselves but instead share the same subtree 
4607 as object A. The example of Fig. 46 causes a problem since the small objects 4601- 
4605 can not be handled separately from object A. As A', opacity quadtree (not 
iUustrated) is propagated upwards, it needs to be successively merged with the opacrty 
information of each of the five small objects, 4601-4605. Representing the small objects 
0 using bounding boxes does no, result in any benefit since it <akes tire same amount of 
effort ,o merge a quadtree with a bounding box as it does to merge two quadtrees. 

A solution ,o me problem of me example of Fig. 46 is <o rearrange .he GOB tree 
4600 into the more favourable structure depicted in Fig. 45, but this raises other rssues. 
Firstly, rearranging the GOB tree 4600 requires an extra traversal of the GOB tree 4600 to 
,5 do so, which can be time consuming if the GOB tree 4600 is large. More importantly 
however, .he opacity quadtrees (no. illus.ra.ed) determined for .he rearranged GOB tree 
may no. be of much use since .he rearranged GOB tiee will need to revert back ,0 ,.s 
original stiuCure due .o any GOB tree optimisation step. Almough ft. opacity quadtrees 
of leaf nodes will not be affected, .hose determined for intermediate nodes will no longer 

30 be valid. - 
GOB tree optimisation is preferably completed prior to obscurance and 

compositing analysis. Since .he GOB tree optimisation meftod describe* above .ends .o 

produce righ, leaning trees, i. is likely ma. the .ype of GOB tree 4600 depicted in F,g. 46 
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would occur quite frequently. It is thus essential for us to provide an effective method for 
handling these cases. 

1 1.2 Processing Small Objects in Right Leaning GOB Trees 

Consider the GOB tree 4700 of Fig. 47, where there is a large object Q at the 
5 bottom of a right biased chain. Suppose that the opacity quadtree for the node *B' at the 
top of the chain needs to be determined. This would require that the opacity quadtrees of 
all nodes in the chain be determined and propagated upwards. As already discussed in 
Section 13.1, such an example poses a problem since using bounding boxes rather than 
quadtrees results in very little benefit. However, by taking advantage of the fact that only 

10 the compositing quadtree of the left operand of a binary operation needs to be determined, 
the opacity quadtrees can be propagated to allow the effective use of bounding boxes. 
Since the opacity quadtrees stored at the intermediate nodes simply act as place holders 
for intermediate results generated in the course of propagating the quadtree from each leaf 
node up to node B, these opacity quadtrees need not be determined explicitly. 

15 1 1.3 Propagation of Opacity Information 

A method of propagating opacity quadtrees using bounding boxes will be 
described with reference to the example of Fig. 47. Considering the example of Fig. 47 
further, at the bottom of the GOB tree 4700, instead of computing the opacity quadtree of 
node Oi, its construction can be delayed by storing both the bounding box Bi of the small 

20 left operand and the opacity quadtree Qi of the large right operand *Q\ A flag can be 
stored at node Oi to indicate that the true opacity is obtained by placing the bounding box 
over the quadtree 4700. Moving up one level, the opacity quadtree of node O2 is given by 
B 2 over (Bi over Qi). But this is equivalent to (B 2 over Bi) over Qi. From the separate 
bounding box B t and quadtree Qi propagated from node Oi, the combined bounding box 

25 B 2 over Bi B 2 u Bi) can be determined, and stored along with the unprocessed 
quadtree Qi at node 0 2 . Therefore, at each node of the chain, a new bounding box can be 
merged with the propagated bounding box. At the top of the chain (i.e. node B), a merge 
between the propagated bounding box and quadtree can be performed to obtain the 
required opacity quadtree for the node B. The right biased tree has been effectively 

30 transformed into a structure of the type shown in Fig. 46, without explicitly rearranging 
the GOB tree 4700. 

The example of Fig. 47 is a somewhat idealised example in that it assumes that 
the opacity of each left operand encountered as the tree 4700 is traversed upward is 
represented solely by a bounding box. If a left operand 4801 along a GOB tree 4800 is a 
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10 



15 



20 



25 



30 



large object, as shown in Fig. 48, then the opacity of the left operand 4801 needs to be 
represented by an opacity quadtree instead of a bounding box. In such a situation, the 
implicit bounding box-quadtree opacity combination can not be propagated all the way up 
to the node B, but rather the true opacity quadtree has to be explicitly determined mid 
way through a GOB tree traversal, upon reaching the large left operand 4801. 

In the example of Fig. 48, node 0 3 contains a large left operand 4801 whose 
opacity is represented by the opacity quadtree Q 3 . The opacity of its right operand 0 2 is 
represented implicitly as [B 2 u B,] over <Q,>, where the expression B 2 u B, within 
square brackets is a rectangular bounding box and the term Q, within angle brackets is an 
opacity quadtree. The opacity of node 0 3 is therefore Q 3 over (B 2 u B,) over Q,. Since 
the expression Q 3 over (B 2 u B,) over Q, cannot be represented as a combination of a 
bounding box and an opacity quadtree, it must be determined explicitly. Once the opacity 
quadtree for node 0 3 is determined, it can be propagated to the node 0 4 - 

The effectiveness of the method of propagating opacity quadtrees using 
bounding boxes is dependent on how large objects are intermixed with small objects. In 
the example of Fig. 48, the presence of a large object results in more quadtree-to-quadtree 
and quadtree-to-bounding box merge operations being performed than if the object was 
small. Nevertheless, the method of propagating opacity quadtrees using bounding boxes 
is still more efficient than having to explicitly compute the opacity quadtrees of all nodes. 
The same situation arises when a left operand is a complex expression whose opacity is 
represented by a quadtree. It should be noted that the opacity of a complex left operand 
can never be represented implicitly as a combination of a bounding box and a quadtree 
since it needs to be explicitly determined to construct the compositing quadtree required 

tcrcomposite the operand. 

Another situation that can result in the computation of opacity quadtrees at 
intermediate nodes during the propagation of opacity quadtrees using the bounding box 
method is where an operator that features in the implicit opacity information of the right 
operand differs from the parent node's operator. Since the propagation of opacity can 
also be applied to other associative operators, namely rover, plusc m&plusw, (N.B. it will 
be shown that the method of propagating opacity quadtrees using bounding boxes does 
not need not be applied to in and rin), then it is possible that an operator featured m the 
right operand is different from the parent node's operator. 

Table 12, below, lists the result of propagating each possible combination of 
opacity representations of the left and right operands through the over operator utilising 
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the method of propagating opacity quadtrees and bounding boxes. The left column of the 
Table 12 represents possible forms of the opacity of the right operand. The middle 
column lists corresponding opacity of the parent node when the opacity of the left 
operand is a bounding box B L . Similarly, the right column lists the corresponding opacity 
5 of the parent node when the left operand's opacity is a quadtree Q L . Expressions within 
square brackets '[]' denote bounding boxes and those within angle brackets *<>' denote 
quadtrees. The absence of a square bracket or angle bracket signifies that the operand's 
opacity is represented solely by a bounding box or by a quadtree. The term R represents 
the true opacity of the right operand and its presence indicates that the associated opacity 
10 needs to be explicitly evaluated. Similarly, a complex term within angle brackets 
represents an opacity quadtree obtained by explicitly merging the objects that appear in 
the term. The operator plus denotes either the plusc or plusw operator. 



Table 12 



R 


Resultant when L=[BJ 


Resultant when L=<Ql> 


[Br] 
<Qr> 

[Br] over <Q R > 
[B R ] rover <Q R > 
[B R ] plus <Q R > 


[B L uB R ] 
[B l ] over <Q R > 
[B L v Br] over <Q R > 
[BJ over <R> 
[BJ over <R> 


[Br] rover <Q L > 

<Q L over Q R > 

<Ql over R> 

[B R ] rover <Q L over Q R > 

<Q L over R> 



15 Tables similar to Table 12 can be determined for the operators rover, plusc, and 

plusw. It is not necessary to have a table for in and rin 9 since of their property that if one 
operand is small, then the other operand as well as the parent must also be small. Thus 
the opacity of all nodes in a chain of inlrin operators are preferably represented solely by 
bounding boxes or solely by opacity quadtrees, and hence can be explicitly determined. 

20 1 1.4 Selective Merging of Bounding Boxes 

Section 11.3 described how small objects can be efficiently handled by storing 
opacity information implicitly as a combination of a bounding box and a quadtree. A 
method of propagating opacity quadtrees using boudning boxes was described that can 
process small objects together using bounding box computation instead of quadtrees, even 

25 though these small objects do not appear by themselves in their own subtree. The method 
of propagating opacity quadtrees using bounding boxes described in section 11.3, 
assumes that it is always more beneficial to process bounding boxes together whenever 
possible. This is not always the case, even though bounding boxes can be very easily 
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combined. For example, consider the quadtree 4900 shown in Fig. 49(a). Using the 
method of propagating opacity quadtrees using bouding boxes described in Section 11.3, 
the opacity of node Y is [B 2 u Bi] over <Q>. That is, the opacity of node Y is made up 
of a quadtree Q, and a bounding box which has a rectangular outline of the union of two 
5 small bounding boxes Bi and B 2 . Preferably, Bi and B 2 are not combined into a single 
bounding box since the rectangular union 4901 of two small rectangles is not necessarily 
small when the rectangles are far apart, as shown in Fig. 49(b). In the example of Fig. 
49(b), it would be detrimental to represent the opacity of the combined rectangles using a 
bounding box, since much of the space inside the box would be interpreted as non-empty, 
10 and hence full compositing operations would then need to be performed there. In contrast 
a better method would be to adopt a quadtree to represent the opacity of the combined 
rectangles. 

For the example of Fig. 49, it is not necessary to obtain the explicit opacity 
quadtree for node Y, which would require two quadtree merge operations. It is sufficient 
15 to compute Bi over Q in a single merge operation and then represent Y's opacity as [B 2 ] 
over <Bi over Q>. Therefore, there is now less area of the resulting object to composite. 

Since compositing time varies with the area being composited, given two 
possibly overlapping bounding boxes Bi and B 2 , the extra time needed to composite their 
rectangular union as compared to compositing their actual areas is proportional to 

20 

Area(Bi kj B 2 ) - Area(Bi) - Area(B 2 ) + Area(Bi n B 2 ) 

This time can be saved when Y's opacity is represented as '[B 2 ] over <Bi over Q>* as 
described above. However, an extra quadtree-to-bounding box merge operation is needed 
25 to determine <Bi over Q>, requiring a processing time that is approximately proportional 
to the perimeter of the bounding box Bi. A possible test to determine whether to 
composite the rectangular union 4901 of Bi and B 2 as compared to compositing their 
actual areas is given by the following comparison: 

30 Area(Bi u B 2 ) - Area(Bi) - Area(B 2 ) + Area(Bi r\ B 2 ) > A,.Perimeter(B) 

where A. is some constant. If the above expression is true, then node Y's opacity is 
represented as [B 2 ] over <B\ over Q>, otherwise it is represented as [B 2 *^ Bi] over <Q>. 
12.0 Propagation* of Compositing Quadtrees 
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Unlike the computation of opacity information, it is possible to compute only 
partial information when it comes to propagating compositing quadtrees. As described 
above, when performing a compositing operation, the compositing quadtree of the left 
operand is required to identify how and where the object needs to be composited. It is not 
5 necessary to compute the compositing quadtree for the right operand. However, the right 
operand can have children and hence would need to collate the obscurance information 
passed down from its parent, and any additional obscurance due to its sibling, and pass 
this down to its own children. Since obscurance information can be encapsulated within 
compositing quadtrees, the compositing quadtree for the right operand may need to be 

10 generated as well. 

The information that needs to be propagated down from a parent node to its 
children is obscurance information, rather than compositing information. Thus, 
preferably obscurance quadtrees are propagated from a parent node to its children as in 
the obscurance analysis described in sections 1.0 to 10.0, rather than propagating 

15 compositing quadtrees. However, since compositing quadtrees also carry the necessary 
obscurance information, propagating compositing quadtrees avoids the need to construct 
separate obscurance quadtrees. 

Where a compositing quadtree is not needed, an obscurance quadtree is 
sufficient for the purpose of propagating the necessary obscurance information. An 

20 obscurance quadtree can be viewed as a compositing quadtree with only partial 
information, and can be constructed with less effort. Constructing an obscurance 
quadtree for a node generally involves the obscurance/compositing quadtree of the parent 
and the opacity quadtree of its sibling, whereas constructing a compositing quadtree also 
involves the node's own opacity quadtree. In other words, constructing an obscurance 

25 quadtree generally requires one less quadtree merge operation. 

Instead of using the node values *v* and c h' to represent visible and hidden 
regions, as previously described for obscurance quadtrees, when performing compositing 
analysis the same node values used in compositing quadtrees are used . Hidden regions 
are assigned the node value *-\ whilst visible regions are assigned the node value V or 

30 T. Therefore, a compositing quadtree can play the role of an obscurance quadtree 
without having to convert between node values. 

The hidden *-* region of the obscurance quadtree for the right operand of each 
binary operator is given in Table 13 below. Q P is the compositing/obscurance quadtree of 
the parent node, whilst Q L > Qr are the opacity quadtrees of the left and right operands 
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respectively. It can be seen from Table 13 that for the most common operators over and 
rover, the obscurance quadtree of the right operand does not depend on its own opacity 
quadtree. The compositing quadtree of the right operand on the other hand, depends on 
the opacity quadtrees of both itself and its sibling. 

Table 13 



Operator 


Right operand's hidden region 


over, rout 

rover, plusc, plusw, rops 
in, out 
rin, ratop 
atop, xor 


{Q P = <-'}u{Q L ='o'} 
{Qp = € - 1 } 

{Q P = '-'}u{Q R = 'o'or Ql='-'} 

{Q P ='-'}u{Q L ='-'} 

{Qp = '-'} u {Q L = 'o' and Q R = V} 



The benefit of propagating obscurance instead of compositing quadtrees lies in 
the processing of small objects. Consider the GOB tree 5000 shown in Fig. 50 where a 
series of small objects appear in a right biased chain of over operations. Assume that 

10 obscurance information in the form of a compositing/obscurance quadtree Q P is to be 
propagated from the top of the GOB tree 5000 to the large object Q at the bottom of the 
tree 5000. Obscurance/compositing analysis is not required for small objects so the 
obscurance information does not need to be propagated down the left side of the GOB 
tree 5000. Using the bounding box method of opacity quadtree propagation described 

15 above, the opacity information of intermediate nodes O1-O3 has been represented 
implicitly as combinations of quadtrees and bounding boxes as shown in the Fig. 50. 

Now if the compositing quadtrees were being propagated down the tree 5000, 
then the true opacity of each of these nodes O3 -Oi needs to be determined since in order 
to compute the compositing quadtree of a node, its opacity quadtree must be explicitly 

20 available. But this would defeat the purpose of representing small objects using bounding 
boxes. 

On the other hand, it can be seen from Table 13 that the computation of 
obscurance quadtrees for the above intermediate nodes O1-O3 does not depend on their 
opacity quadtrees, and hence they do not need to be explicitly evaluated. In fact since the 
25 left siblings of the nodes O1-O3 are small objects, which do not contain any fully opaque 
regions, the definition of the obscurance quadtrees of these nodes reduces from {Qp = '-'} 
yj {Q L = v} to {Q P = '-'}• Thus the quadtree Q P passed down from the top of the chain 
can simply be propagated down unaltered. 
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The same advantage of propagating obscurance rather than compositing 
quadtrees also applies to the other associative operators that can possess implicit opacity 
information, namely rover ; plusc and plusw. 

In situations where a GOB tree does not include an uninterrupted chain of small 
5 objects, but instead has a large object in the middle of the chain as shown in Fig. 48, then 
some adjustment is necessary. Although it is still valid to propagate obscurance quadtrees 
down to each right operand along the chain, the need to compute the compositing 
quadtree for the large object in the middle of the chain would mean that the opacity of 
the object's right sibling would need to be determined since a compositing quadtree of 

10 any node is dependent on the opacity quadtrees of not only the node concerned but also 
its sibling. Thus in Fig. 48, the opacity of node O2 would need to be explicitly 
determined. The opacity of the remaining nodes however can still be represented 
implicitly as described above. 
12.1 Testing for Fully Obscured Objects 

15 Compositing analsyis as described in the previous sections only allows for partial 

testing of totally obscured objects. Such an object can be identified from its compositing 
quadtree comprising of a single empty node Since compositing quadtrees are only 
determined for the left operands of binary operators, only the left operands of binary 
operators can be tested for total obscurance. 

20 The obscurance quadtrees determined for the right operands only contain 

information about regions that are hidden, which as currently defined, exclude regions 
where the operands are empty. In order to detect total obscurance, regions that are either 
hidden or empty need to be detected. Although the computation of obscurance quadtrees 
can be modified to detect total obscurance, doing so increases an obscurance quadtree's 

25 computational complexity. Considering that compositing analysis can detect totally 
obscured left operands in addition to those detected during bounding box minimisation, 
and the fact that undetected objects will not be composited at the final rendering stage, 
such a modification to obscurance quadtrees is not worthwhile. 
13.0 Compositing with Small Objects 

30 For compositing analysis to produce correct outputs, it is essential when 

rendering objects that the obscurance/compositing information given to the objects is 
strictly obeyed. This means that if an object is hidden in a particular region, as dictated 
by its obscurance quadtree, then it should not be rendered in that region. Similarly, if an 
object is to be rendered using a load operation in a particular region, then it must not be 
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rendered using compositing. By the same reason, when rendering a primitive, its 
boundary must be padded with transparent pixels to the outline indicated by the 

primitive's opacity quadtree. 

In using compositing analysis, preferably rendering should be pixel-accurate. 
That is, two adjacent rectangular regions must be rendered without any visible pixel gaps 
since overlapping objects can now be rendered on a piece-wise basis, rather than by 
depositing one fully formed object on top of another fully formed object. 

Small objects are preferably treated differently in compositing analysis since 
there should not be a need to compute the small object's opacity/obscurance/compositing 
quadtrees. Instead their opacity can be represented by bounding boxes. 

It would be invalid to render an object without giving regards to its obscurance 
information. However, this does not necessarily mean that a new obscurance quadtree for 
each object in the GOB tree needs to be determined. Instead, in most situations where a 
small object is encountered, its obscurance/compositing quadtree is identical to the 
obscurance quadtree propagated down from its parent. There are three possible scenarios 
involving binary composition of small objects, namely when the left, right or both 
operands are small. 

13.1 Compositing with a small right operand 

Consider the binary operation L in R shown in Fig. 32, where object R is small. 
If object R is rendered in its entirety and then composited onto by the left operand L, then 
the region of R 3200 that falls outside of L will remain after the operation, which is 
incorrect since preferably the compositing operation is performed only within the outline 
of the left operand, rather than over the entire render space. This problem does not occur 
if the right operand is rendered according to its obscurance information, which would 
indicate that the operand is invisible outside of L's outline. However, this would require 
that R's obscurance quadtree be determined, which is preferably avoided when R is small. 

Alternatively, the compositing region of the left operand can be enlarged beyond 
its outline to encompass R. However, the most effective solution is to compute an 
obscurance quadtree for the right operand even if it is a small object. 

The above problem is only present with binary operators in, rin, out and ratop. 
These operators have the special property that the parts of the right operands that do not 
intersect with the left are altered by the application of the operators. Since small objects 
are usually not composited using the above operators, the need to compute their 
obscurance quadtrees does not arise too frequently. 
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For other operators, a small right operand can be rendered without the need to 
consider any obscurance from its left sibling. If the operator involved in the example of 
Fig. 32 were 'over' instead of then the hidden part of R would preferably 

automatically be overwritten by L. The obscurance quadtree passed down from the 
5 parent node needs to be considered and the primitive in the hidden regions need not be 
rendered. However, this does not require a new obscurance quadtree to be determined for 
the right operand. 

Fig. 33(a) shows the propagation and computation of compositing/obscurance 
information when a large left operand is composited onto a small right operand where the 

10 right operand's obscurance quadtree Qp is identical to the parents. Fig. 33(b) shows the 
computation of compositing/obscurance information when a large left operand is 
composited onto a small right operand for an in operator where the obscurance quadtree 
needs to be determined from the parent node's opacity quadtree and the left operand's 
bounding box. Regardless of whether the right operand is small or not, a compositing 

15 quadtree needs to be determined for the left operand to identify regions that can be 
rendered using a load operation, and regions that require full compositing. 

13.2 Compositing with a Small Left Operand 

When the left operand of a compositing operation is small, the left operand is 
preferably rendered by performing a full compositing operation over its bounding box 
20 after the bounding box has been reduced by the parent's obscurance quadtree. 

In addition to identifying how an operand is rendered, a compositing quadtree 
can perform as an obscurance quadtree when the operand is complex. The obscurance 
quadtree is propagated down to the child nodes below. In the case where the left operand 
is small and hence its compositing quadtree is not determined, its obscurance quadtree is 
25 taken to be the parent's obscurance quadtree, unless the reduced compositing region 
described in the preceding paragraph is empty. 

The obscurance quadtree for the right operand is determined from its opacity 
quadtree and the left operand's bounding box. This is not necessary for operators such as 
over since the left operand is small and can not obscure any part of the right operand. It is 
30 necessary however, for operators such as in and out that have 'clipping' actions. 

13.3 Compositing with two small operands 

When compositing with two small operands, since both operands are small, any 
obscurance that may occur between the operands can be disregarded. Any 'clipping' 
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actions between the operands can also be disregarded due to operators such as in and out, 
since the clipping operations are preferably accounted for by bounding box minimisation. 
The right operand's obscurance quadtree is thus identical to the parent's obscurance 
quadtree, and the left operand's compositing region is the intersection of its bounding box 
and the visible region in the parent's obscurance quadtree. 

14.0 Using Compositing Quadtrees to Limit the Processing when Rendering 
Images 

14.1 Preliminary Processing 

Preferably, compositing analysis is carried out after any GOB tree optimisation 
methods such as rearranging the GOB tree. Compositing analsysis is most beneficial 
when applied to the final GOB tree prior to rendering, since the resulting compositing 
quadtrees are particular to the structure of the GOB tree in which they are produced. 

Further, compositing analysis preferably occurs after bounding box 
minimisation, since the later is a prerequisite to GOB tree optimisation, where bounding 
boxes are used to determine the complexity of each primitive and composite object. 
Thus, bounding boxes obtained through bounding box minimisation can be utilised as a 
means for identifying small objects, for both primitive and complex objects. 
14.2 Preferred Method for Propagating Compositing Quadtrees 

Fig. 51 is a flowchart showing a method of propagating compositing quadtrees in 
order to optimise a GOB tree representing an image. The method of Fig. 51 comprises 
two traversals of the GOB tree. The first traversal is a bottom up process in which 
opacity information in the form of opacity quadtrees and bounding boxes are generated 
andrpropagated upwards. In the second traversal, compositing/obscurance quadtrees are 
produced and propagated downwards. 

In compositing analysis, as opposed to obscurance analysis, almost every node in 
the GOB tree not associated with small objects requires it's opacity information to be 
determined. As described in section 14.0, compositing quadtrees do not need to be fully 
evaluated for every node. For certain nodes, it is sufficient to compute the obscurance 
quadtrees. For other nodes, no compositing/obscurance quadtrees need to be determined. 
Details on which nodes need to have their opacity/compositing/obscurance information 

evaluated are described below. 

The process begins at step 5101, where a bottom-up traversal of the nodes of the 
GOB tree is started. At the next step 5103, when a node is encountered, if the node is 
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* small', the the entire subtree below the node can be ignored, since every node in that 
subtree would also be small, and hence no opacity information is required for the subtree. 
A primitive or complex object is considered small if the size of its bounding box is below 
a predetermined threshold. This threshold is determined experimentally. If the node is 
5 small, the process continues to step 5105 and the traversal proceeds to the next node. 
Otherwise, the process proceeds to step 5107, where opacity information in the form of 
bounding boxes and quadtrees is generated and propagated upwards. The rules governing 
the computation of opacity information depend on whether the node is a primitive or a 
non- primitive and are as follows: 
10 Primitives 

(i) If a primitive is small, then its opacity is represented by its bounding box, 
otherwise an opacity quadtree is constructed. 

Non Primitives 

15 

(i) The opacity of a complex left operand of a binary operator must be evaluated 
explicitly either as a bounding box if the operand is small, or as a quadtree but not as a 
combination of both. The opacity is required to compute the operand's compositing 
quadtree. 

20 (ii) The opacity of a complex right operand of a binary operator must be evaluated 
explicitly if its left sibling requires a compositing quadtree. 

(iii) The opacity of the complex child operand of a convolution/gradient/unsharp 
mask operator must be represented either as bounding box or a quadtree, but not as a 
combination of both. The opacity is required to perform the necessary processing on the 

25 opacity as a result of the convolution operation. 

(iv) If the opacity of one or both operands of a binary operator contain a bounding 
box, then Table 12 dictates the form of the resulting opacity of the parent node. If an 
operand's opacity is implicitly represented as a combination of a bounding box and a 
quadtree, then Table 12 also dictates cases where the information needs to be explicitly 

30 evaluated. 

At the next step 5109, the node is tagged as to whether its 
compositing/obscurance quadtree needs to be determined in the next GOB tree traversal. 
The process continues at step 5111, where if the bottom-up traversal has not finished the 
process returns to step 5105, where the traversal proceeds to the next node. Otherwise, 
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the process proceeds to the next step 5113, where a top-down traversal of the GOB tree is 
begun starting at the root node of the GOB tree and proceeding to the first node. The root 
node's obscurance quadtree comprises a single V node, signifying that this node is 
visible everywhere. At each node, a decision is made as to whether a compositing or an 
5 obscurance quadtree needs to be determined. At the next step 51 15, if the node is the left 
operand of a binary operator and its bounding box is large, then the process proceeds to 
step 5117 where the node's compositing quadtree is determined. Otherwise, the process 
continues to the next step 5119, where if the node is large and it is the right operand of a 
binary operator, then the process proceeds to step 5121, where the node's obscurance 

10 quadtree is determined. Otherwise, the process continues to the next step 5123, where if 
the node is small and it is the right operand of a binary in, rin, out or ratop operator, then 
the process proceeds to step 5125, where the node's obscurance quadtree is determined. 
Otherwise, the process continues to the next step 5127, where if the node is a child 
operand of a unary operator, then the process proceeds to step 5129, where the node's 

15 obscurance quadtree is computed. A compositing quadtree is not needed for unary 
operators since no compositing operations are performed. Otherwise, the process 
continues to the next step 5131, where the node's parents' obscurance quadtrees are 
propagated further down. The obscurance and compositing quadtrees determined at one 
of steps 5117, 5121, 5125 and 5129 for the node are passed down to any children of the 

20 node at the next step 5133. The process continues at the next step 5135, where if the top- 
down traversal of the GOB tree has not finished, then the process continues to step 5137, 
where the traversal proceeds to the next node. Otherwise the process concludes. 

The aforementioned preferred methods comprise a particular control flow. There 
are" many other variants of the preferred methods which use different control flows 

25 without departing the spirit or scope of the invention. Furthermore one or more of the 
steps of the preferred methods may be performed in parallel rather sequential. 
15,0 Preferred Apparatus 

The methods described above are preferably practiced using a conventional 
general-purpose computer system 2900, such as that shown in Fig. 29 wherein the 

30 processes of Figs. 1 to 28 and 30 to 51 can be implemented as software, such as an 
application program executing within the computer system 2900. In particular, the steps 
of the methods can be effected by instructions in the software that are carried out by the 
computer. The software may be divided into two separate parts; one part for carrying out 
the methods; and another part to manage the user interface between the latter and the user. 
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The software may be stored in a computer readable medium, including the storage 
devices described below, for example. The software is loaded into the computer from the 
computer readable medium, and then executed by the computer. A computer readable 
medium having such software or computer program recorded on it is a computer program 
5 product. The use of the computer program product in the computer preferably effects an 
advantageous apparatus for performing the methods described above. 

The computer system 2900 comprises a computer module 2901, input devices 
such as a keyboard 2902 and mouse 2903, output devices including a printer 2915 and a 
display device 2914. A Modulator-Demodulator (Modem) transceiver device 2916 is 

10 used by the computer module 2901 for communicating to and from a communications 
network 2920, for example connectable via a telephone line 2921 or other functional 
medium. The modem 2916 can be used to obtain access to the Internet, and other 
network systems, such as a Local Area Network (LAN) or a Wide Area Network (WAN). 

The computer module 2901 typically includes at least one processor unit 2905, a 

15 memory unit 2906, for example formed from semiconductor random access memory 
(RAM) and read only memory (ROM), input/output (I/O) interfaces including a video 
interface 2907, and an I/O interface 2913 for the keyboard 2902 and mouse 2903 and 
optionally a joystick (not illustrated), and an interface 2908 for the modem 2916. A 
storage device 2909 is provided and typically includes a hard disk drive 2910 and a 

20 floppy disk drive 291 L A magnetic tape drive (not illustrated) may also be used. A CD- 
ROM drive 2912 is typically provided as a non- volatile source of data. The 
components 2905 to 2913 of the computer module 2901, typically communicate via an 
interconnected bus 2904 and in a manner which results in a conventional mode of 
operation of the computer system 2900 known to those in the relevant art. Examples of 

25 computers on which the embodiments can be practised include IBM-PC's and 
compatibles, Sun Sparcstations or alike computer systems evolved therefrom. 

Typically, the application program is preferably resident on the hard disk 
drive 2910 and read and controlled in its execution by the processor 2905. Intermediate 
storage of the program and any data fetched from the network 2920 may be accomplished 

30 using the semiconductor memory 2906, possibly in concert with the hard disk drive 2910. 
In some instances, the application program may be supplied to the user encoded on a CD- 
ROM or floppy disk and read via the corresponding drive 2912 or 291 1, or alternatively 
may be read by the user from the network 2920 via the modem device 2916. Still further, 
the software can also be loaded into the computer system 2900 from other computer 
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readable medium including magnetic tape, a ROM or integrated circuit, a magneto-optical 
disk, a radio or infra-red transmission channel between the computer module 2901 and 
another device, a computer readable card such as a PCMCIA card, and the Internet and 
Intranets including email transmissions and information recorded on websites and the 
like. The foregoing is merely exemplary of relevant computer readable mediums. Other 
computer readable mediums may be practiced without departing from the scope and spirit 
of the invention. 

The methods described above can alternatively be implemented in dedicated 
hardware, such as one or more integrated circuits performing the functions or sub 
functions of of Figs. 1 to 28 and 30 to 51. Such dedicated hardware may include graphic 
processors, digital signal processors, or one or more microprocessors and associated 
memories. 

The foregoing describes only some embodiments of the present invention, and 
modifications and/or changes can be made thereto without departing from the scope and 
spirit of the invention, the embodiments being illustrative and not restrictive. For 
example, bounding boxes can be used to represent opacity, obscurance and compositing 
information for an image, in accordance with the above described embodiments. 

In the context of this specification, the word "comprising" means "including 
principally but not necessarily solely" or "having" or "including" and not "consisting only 
of*. Variations of the word comprising, such as "comprise" and "comprises" have 
corresponding meanings. 
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1. A method for optimising an expression tree, said expression tree representing a 
compositing expression for compositing an image and comprising a plurality of nodes, 
each said node of said tree representing at least a component of said image or an 
5 operation for combining sub-expressions of said compositing expression and each said 
node having a region of the image represented by said node, the method comprising the 
steps of: 

performing a first traversal of said tree to determine opacity information for each 
node of said tree, wherein said opacity information represents combined opacity 
10 information for a node based on regions associated with the node; and 

optimising said expression tree by performing a second traversal of said tree to 
determine compositing information for at least one node of said tree, wherein said 
compositing information for a node is determined using opacity information associated 
with the node, and wherein said compositing information represents at least obscured 
15 regions, load regions and regions to be composited for an object associated with the node. 

1A. The method according to claim 1, wherein said compositing information is 
represented by a first hierarchical structure. 

20 2. The method according to claim 1A, further comprising the step of identifying 

nodes of said tree, for which a first hierarchical structure is required, depending on 
opacity information associated with the node. 

3. The method according to claim 1, wherein said opacity information comprises a 
25 second hierarchical structure representing an opacity of a region associated with a node. 

4. The method according to claim 1 , wherein said opacity information comprises a 
bounding box representing an opacity of a region associated with a node. 

30 5. The method according to claim 1A, wherein said said first hierarchical structure 

is dependent on said opacity information. 
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6. The method according to claim 1A, wherein said first traversal is a bottom-up 

traversal. 

5 7. The method according to claim 3, wherein opacity information of a child node is 

at least propagated to a parent node associated with said child node. 

8. The method according to claim 7, wherein opacity information of said parent 
node is determined by merging at least two second hierarchical structures. 

10 

9. The method according to claim 7, wherein opacity information of said parent 
node is determined by merging at least one second hierarchical structure and a bounding 
box. 

15 10. The method according to claim 1, wherein said second traversal is a top-down 
traversal. 

11. The method according to claim 8, wherein a first hierarchical structure of a 
parent node is at least propagated to a child node associated with said parent node. 

20 

12. The method according to claim 3, wherein said second hierarchical structure is 
dependent on an operation associated with a node for which said first hierarchical 
structure is constructed. 

25 13. The method according to claim 3, wherein each leaf node of said second 
hierarchical structure is assigned a value depending on an opacity of a region associated 
with said leaf node. 

14. The method according to claim 3, wherein each node of said second hierarchical 
30 structure comprises a pointer to indicate child nodes associated with said node. 
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15. The method according to claim 3, wherein said first and second hierarchical 
structures are quadtrees. 

5 15 A. A method for optimising an expression tree, said expression tree representing a 
compositing expression for compositing an image and comprising a plurality of nodes, 
each said node of said tree representing at least a component of said image or an 
operation for combining sub-expressions of said compositing expression and each said 
node having a region of the image represented by said node, the method comprising the 
10 steps of: 

performing a first traversal of said tree to determine opacity information for each 
node of said tree, wherein said opacity information represents combined opacity 
information for a node based on regions associated with the node; and 

optimising said expression tree by performing a second traversal of said tree to 
15 construct a first hierarchical structure for at least one node of said tree, wherein said first 
hierarchical structure is constructed for a node using opacity information associated with 
the node, and wherein said first hierarchical structure represents at least obscured regions, 
load regions and regions to be composited, for an object associated with the node.. 

20 16. A method for optimising an expression tree, said expression tree representing a 
compositing expression for compositing an image and comprising a plurality of nodes, 
each said node of said tree representing at least a component of said image or an 
operation for combining sub-expressions of said compositing expression and each said 
node having a region of the image represented by said node, the method comprising the 

25 steps of: 

performing a first traversal of said tree to determine opacity information for each 
node of said tree, wherein said opacity information represents combined opacity 
information for a node based on regions associated with the node; 

identifying nodes of said tree, for which a first hierarchical structure is required, 
30 depending on opacity information associated with the node; and 

optimising said expression tree by performing a second traversal of said tree to 
determine compositing information for each node of said tree - identified in said first 



I:\501203.doc 



-79- 



O 



traversal, wherein said compositing information is determined for a node using opacity 
information associated with the node, and wherein said compositing information 
represents at least obscured regions, load regions and regions to be composited for an 
object associated with the node. 

5 

16A. The method according to claim 16, wherein said compositing information is 
represented by a first hierarchical structure. 

17. The method according to claim 16 A, wherein said opacity information comprises 
10 a second hierarchical structure representing an opacity of a region associated with a node. 

18. The method according to claim 16, wherein said opacity information comprises 
a bounding box representing an opacity of a region associated with a node. 

15 19. The method according to claim 16A, wherein said first hierarchical structure is 
dependent on said opacity information. 

20. The method according to claim 16, wherein said first traversal is a bottom-up 
traversal. 

20 

21. . The method according to claim 17, wherein opacity information of a child node 
is at least propagated to a parent node associated with said child node. 

22. The method according to claim 21, wherein opacity information of said parent 
25 node is determined by merging at least two second hierarchical structures. 

23. The method according to claim 21, wherein opacity information of said parent 
node is determined by merging at least one second hierarchical structure and a bounding 
box. 

30 
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24. The method according to claim 16A, wherein said second traversal is a top-down 
traversal. 

25. The method according to claim 24, wherein a first hierarchical structure of a 
5 parent node is at least propagated to a child node associated with said parent node. 

26. The method according to claim 17, wherein said second hierarchical structure is 
dependent on an operation associated with a node for which said first hierarchical 
structure is constructed. 

10 

27. The method according to claim 17, wherein each leaf node of said second 
hierarchical structure is assigned a value depending on an opacity of a region associated 
with said leaf node. 

15 28. The method according to claim 17, wherein each node of said second 
hierarchical structure comprises a pointer to indicate child nodes associated with said 
node. 

29. The method according to claim 17, wherein said first and second hierarchical 
20 structures are quadtrees. 

30. An apparatus configured to perform the method according to any one of claims 1 
to 28. 

25 31. A computer readable medium for storing a program for an apparatus which 
processes data, said processing comprising a process for optimising an expression tree, 
said program comprising code for performing the method according to any one of claims 
1 to 29. 

30 DATED this Thirty- first Day of May 2000 

Canon Kabushiki Kaisha 
Patent Attorneys for the Applicant 
SPRUSON & FERGUSON 
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